[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:43 dist-test-slave-sm0p krb5kdc[13381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:43 dist-test-slave-sm0p krb5kdc[13381](info): set up 2 sockets
Sep 29 06:28:43 dist-test-slave-sm0p krb5kdc[13381](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 06:28:44.774668 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:44 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:44.889286 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37579
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:44.966593 13394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:44.966804 13394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:44.966840 13394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:44.967885 13394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:44.967947 13394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:44.967986 13394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:44.968012 13394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:44.968039 13394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:44.969403 13394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:44.969641 13394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:44.969856 13394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:44.974992 13399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:44.974998 13400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:44.975016 13402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:44.975080 13394 server_base.cc:1034] running on GCE node
I20260929 06:28:44.975596 13394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:44.975907 13394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:44.977105 13394 hybrid_clock.cc:648] HybridClock initialized: now 1790663324977092 us; error 27 us; skew 500 ppm
Sep 29 06:28:44 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:44.979540 13394 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:44.979954 13394 webserver.cc:460] Webserver started at http://127.13.15.190:39171/ using document root <none> and password file <none>
I20260929 06:28:44.980167 13394 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:44.980232 13394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:44.980376 13394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:44.981436 13394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "5d540debdbee400984d766cd60dcd904"
format_stamp: "Formatted at 2026-09-29 06:28:44 on dist-test-slave-sm0p"
server_key: "aeb685b50e47bc116e21a148af3ff0f6"
server_key_iv: "ce0a26db19bc4860820999e1a3eedfb0"
server_key_version: "encryptionkey@0"
I20260929 06:28:44.981789 13394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "5d540debdbee400984d766cd60dcd904"
format_stamp: "Formatted at 2026-09-29 06:28:44 on dist-test-slave-sm0p"
server_key: "aeb685b50e47bc116e21a148af3ff0f6"
server_key_iv: "ce0a26db19bc4860820999e1a3eedfb0"
server_key_version: "encryptionkey@0"
I20260929 06:28:44.983263 13394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:44.983973 13409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:44.984140 13394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:44.984236 13394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "5d540debdbee400984d766cd60dcd904"
format_stamp: "Formatted at 2026-09-29 06:28:44 on dist-test-slave-sm0p"
server_key: "aeb685b50e47bc116e21a148af3ff0f6"
server_key_iv: "ce0a26db19bc4860820999e1a3eedfb0"
server_key_version: "encryptionkey@0"
I20260929 06:28:44.984308 13394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:44.994717 13394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:44.996351 13394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:44.996505 13394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:45.000005 13394 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:37579
I20260929 06:28:45.000046 13461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:37579 every 8 connection(s)
I20260929 06:28:45.000407 13394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:45.001220 13462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:45.003333 13462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904: Bootstrap starting.
I20260929 06:28:45.003684 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 13394
I20260929 06:28:45.003787 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:45.003968 13462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:45.004256 13462 log.cc:824] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:45.004475 13374 external_mini_cluster.cc:1428] Setting key 849caf9f246d963b440b8b628515dadc
I20260929 06:28:45.004963 13462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904: No bootstrap required, opened a new log
I20260929 06:28:45.007287 13462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } }
I20260929 06:28:45.007481 13462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:45.007517 13462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d540debdbee400984d766cd60dcd904, State: Initialized, Role: FOLLOWER
I20260929 06:28:45.007640 13462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } }
I20260929 06:28:45.007720 13462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:45.007750 13462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:45.007794 13462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:45.008373 13462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } }
I20260929 06:28:45.008471 13462 leader_election.cc:302] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d540debdbee400984d766cd60dcd904; no voters:
I20260929 06:28:45.008635 13462 leader_election.cc:288] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:45.008695 13467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:45.008863 13467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [term 1 LEADER]: Becoming Leader. State: Replica: 5d540debdbee400984d766cd60dcd904, State: Running, Role: LEADER
I20260929 06:28:45.008865 13462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:45.008981 13467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } }
I20260929 06:28:45.009263 13467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d540debdbee400984d766cd60dcd904. Latest consensus state: current_term: 1 leader_uuid: "5d540debdbee400984d766cd60dcd904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } } }
I20260929 06:28:45.009469 13467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:45.009403 13468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d540debdbee400984d766cd60dcd904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d540debdbee400984d766cd60dcd904" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37579 } } }
I20260929 06:28:45.009853 13468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:45.009954 13476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:45.010458 13476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:28:45.012867 13476 catalog_manager.cc:1347] Generated new cluster ID: 7bfaafdea512450ea8a310aed554f0fa
I20260929 06:28:45.012908 13476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:45.018231 13476 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:45.018929 13476 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:45.021488 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.005265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.13.15.190:37579)
0929 06:28:45.005452 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.005461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.005790 (+ 329us) server_negotiation.cc:407] Connection header received
0929 06:28:45.008036 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.008042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.008248 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.008348 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.010542 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.011394 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.012704 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.013297 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.013397 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.017540 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.017544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.017544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.017554 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.019650 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.020124 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.020127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.020127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.020218 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.020490 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.020490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.020491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.020724 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.020852 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.021142 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.021275 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260929 06:28:45.024895 13476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d540debdbee400984d766cd60dcd904: Generated new TSK 0
I20260929 06:28:45.025051 13476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:28:45.049050 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:45.126853 13489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:45.127029 13489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:45.127050 13489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:45.128139 13489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:45.128182 13489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:45.128208 13489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:28:45.129541 13489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:45.129783 13489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:45.130029 13489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:45.134933 13495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:45.134939 13494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:45.135061 13489 server_base.cc:1034] running on GCE node
W20260929 06:28:45.134939 13497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:45.135344 13489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:45.135646 13489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:45.136825 13489 hybrid_clock.cc:648] HybridClock initialized: now 1790663325136832 us; error 46 us; skew 500 ppm
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:45.139366 13489 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:28:45.139784 13489 webserver.cc:460] Webserver started at http://127.13.15.129:42749/ using document root <none> and password file <none>
I20260929 06:28:45.140019 13489 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:45.140087 13489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:45.140241 13489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:45.141286 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "a84b5391ec8944f0aa239cdae29b0f72"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "4cad8a5e1ee60ba1fc7f048079f932d4"
server_key_iv: "420cee4cab68ea7689e1ab726c0bf4e8"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.141633 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "a84b5391ec8944f0aa239cdae29b0f72"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "4cad8a5e1ee60ba1fc7f048079f932d4"
server_key_iv: "420cee4cab68ea7689e1ab726c0bf4e8"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.142959 13489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:45.143618 13504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.143790 13489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.143883 13489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "a84b5391ec8944f0aa239cdae29b0f72"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "4cad8a5e1ee60ba1fc7f048079f932d4"
server_key_iv: "420cee4cab68ea7689e1ab726c0bf4e8"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.143977 13489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:45.159626 13489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:45.161798 13489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:45.161947 13489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:45.162235 13489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:45.162549 13489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:45.162601 13489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.162642 13489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:45.162668 13489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.168160 13489 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:37515
I20260929 06:28:45.168249 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:37515 every 8 connection(s)
I20260929 06:28:45.168531 13489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:28:45.173312 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 13489
I20260929 06:28:45.173404 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:28:45.173554 13374 external_mini_cluster.cc:1428] Setting key 6687a07434cc218bd6552eaa53d318fe
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:45.178050 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.169246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:43707 (local address 127.13.15.190:37579)
0929 06:28:45.169364 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.169367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.169536 (+ 169us) server_negotiation.cc:407] Connection header received
0929 06:28:45.170029 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.170030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.170041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.170093 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.171012 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.171502 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.172481 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.172701 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.172790 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.175253 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.175254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.175255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.175261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.176592 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.177054 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.177055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.177055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.177104 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.177376 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.177376 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.177376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.177520 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.177571 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.177892 (+ 321us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.177991 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 06:28:45.178368 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.169205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37579 (local address 127.13.15.129:43707)
0929 06:28:45.169407 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:45.169454 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:45.169929 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:45.170189 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:45.170193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:45.170723 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:45.170900 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.170904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.171594 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.171595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.172390 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.172392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.172797 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.172798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.172924 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.173349 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:45.173359 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:45.175170 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:45.176674 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.176680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.176682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.176984 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.177174 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.177175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.177175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.177310 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.177588 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:45.177594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:45.177759 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 06:28:45.177875 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 06:28:45.178072 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260929 06:28:45.178938 13618 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37579
I20260929 06:28:45.179042 13618 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:45.179263 13618 heartbeater.cc:511] Master 127.13.15.190:37579 requested a full tablet report, sending...
I20260929 06:28:45.179761 13426 ts_manager.cc:194] Registered new tserver with Master: a84b5391ec8944f0aa239cdae29b0f72 (127.13.15.129:37515)
I20260929 06:28:45.180505 13426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:43707
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:28:45.197154 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:45.274932 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:45.275112 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:45.275134 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:45.276222 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:45.276260 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:45.276309 13623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:28:45.277750 13623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:45.277981 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:45.278223 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:45.282832 13629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:45.282850 13628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:45.282876 13623 server_base.cc:1034] running on GCE node
W20260929 06:28:45.282840 13631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:45.283195 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:45.283430 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:45.284597 13623 hybrid_clock.cc:648] HybridClock initialized: now 1790663325284577 us; error 36 us; skew 500 ppm
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:45.287110 13623 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:28:45.287500 13623 webserver.cc:460] Webserver started at http://127.13.15.130:42183/ using document root <none> and password file <none>
I20260929 06:28:45.287716 13623 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:45.287781 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:45.287925 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:45.288874 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "859d08dd00514c7797c3c8561b0300ff"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "d5eea03934cb1634039555f938b6471c"
server_key_iv: "03e45b6bab61e8f8a03d291cb59763d6"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.289222 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "859d08dd00514c7797c3c8561b0300ff"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "d5eea03934cb1634039555f938b6471c"
server_key_iv: "03e45b6bab61e8f8a03d291cb59763d6"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.290566 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:45.291322 13638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.291478 13623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.291572 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "859d08dd00514c7797c3c8561b0300ff"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "d5eea03934cb1634039555f938b6471c"
server_key_iv: "03e45b6bab61e8f8a03d291cb59763d6"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.291656 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:45.334180 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:45.336580 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:45.336743 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:45.336972 13623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:45.337270 13623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:45.337322 13623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.337365 13623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:45.337393 13623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.343137 13623 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:44473
I20260929 06:28:45.343232 13751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:44473 every 8 connection(s)
I20260929 06:28:45.343521 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:45.351631 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 13623
I20260929 06:28:45.351725 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:28:45.351925 13374 external_mini_cluster.cc:1428] Setting key ffc48a131ee13c1e29bf7fd3129c6d36
I20260929 06:28:45.352398 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.344239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:49179 (local address 127.13.15.190:37579)
0929 06:28:45.344316 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.344318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.344478 (+ 160us) server_negotiation.cc:407] Connection header received
0929 06:28:45.344917 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.344919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.344928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.344978 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.345865 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.346419 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.347428 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.347661 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.347744 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.349654 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.349655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.349655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.349661 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.350998 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.351342 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.351342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.351343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.351391 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.351666 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.351666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.351666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.351818 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.351877 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.352229 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.352312 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:28:45.352581 13754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.344204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37579 (local address 127.13.15.130:49179)
0929 06:28:45.344364 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:45.344397 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:45.344844 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:45.345028 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:45.345032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:45.345597 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:45.345784 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.345789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.346553 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.346554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.347336 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.347338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.347748 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.347749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.347871 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.348281 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:45.348289 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:45.349567 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:45.351058 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.351063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.351064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.351280 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.351446 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.351447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.351447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.351596 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.351889 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:45.351896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:45.352052 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 06:28:45.352184 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 06:28:45.352335 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260929 06:28:45.353034 13752 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37579
I20260929 06:28:45.353137 13752 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:45.353302 13752 heartbeater.cc:511] Master 127.13.15.190:37579 requested a full tablet report, sending...
I20260929 06:28:45.353696 13426 ts_manager.cc:194] Registered new tserver with Master: 859d08dd00514c7797c3c8561b0300ff (127.13.15.130:44473)
I20260929 06:28:45.354234 13426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:49179
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:28:45.374675 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:45.452311 13757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:45.452503 13757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:45.452540 13757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:45.453742 13757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:45.453804 13757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:45.453851 13757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:28:45.455277 13757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37579
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:45.455518 13757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:45.455746 13757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:45.460413 13763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:45.460412 13762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:45.460438 13765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:45.460731 13757 server_base.cc:1034] running on GCE node
I20260929 06:28:45.460932 13757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:45.461163 13757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:45.462312 13757 hybrid_clock.cc:648] HybridClock initialized: now 1790663325462309 us; error 38 us; skew 500 ppm
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:45.464584 13757 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:28:45.464974 13757 webserver.cc:460] Webserver started at http://127.13.15.131:40131/ using document root <none> and password file <none>
I20260929 06:28:45.465176 13757 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:45.465247 13757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:45.465394 13757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:45.466439 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "70003c70d07f40159a0baf2f4e89117c"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "38c26e014b837ae636f2eb6fb4eac97a"
server_key_iv: "c9ecc4c1b0615a4121b4b8c9528f23be"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.466777 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "70003c70d07f40159a0baf2f4e89117c"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "38c26e014b837ae636f2eb6fb4eac97a"
server_key_iv: "c9ecc4c1b0615a4121b4b8c9528f23be"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.467957 13757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:28:45.468569 13772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.468732 13757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:45.468817 13757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "70003c70d07f40159a0baf2f4e89117c"
format_stamp: "Formatted at 2026-09-29 06:28:45 on dist-test-slave-sm0p"
server_key: "38c26e014b837ae636f2eb6fb4eac97a"
server_key_iv: "c9ecc4c1b0615a4121b4b8c9528f23be"
server_key_version: "encryptionkey@0"
I20260929 06:28:45.468894 13757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:45.487970 13757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:45.489648 13757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:45.489806 13757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:45.490008 13757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:45.490358 13757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:45.490412 13757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.490445 13757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:45.490471 13757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:45.496068 13757 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:46113
I20260929 06:28:45.496137 13885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:46113 every 8 connection(s)
I20260929 06:28:45.496440 13757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:28:45.498672 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 13757
I20260929 06:28:45.498754 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:28:45.498924 13374 external_mini_cluster.cc:1428] Setting key 12e8442b61a950cc1cd8c1459ec0e350
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:45.505326 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.497143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:57081 (local address 127.13.15.190:37579)
0929 06:28:45.497233 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.497235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.497355 (+ 120us) server_negotiation.cc:407] Connection header received
0929 06:28:45.497812 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.497813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.497822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.497888 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.498865 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.499504 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.500550 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.500783 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.500863 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.502804 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.502805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.502805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.502829 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.504050 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.504399 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.504400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.504400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.504451 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.504652 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.504653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.504653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.504757 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.504804 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.505193 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.505264 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 06:28:45.505491 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.497090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37579 (local address 127.13.15.131:57081)
0929 06:28:45.497258 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:45.497290 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:45.497700 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:45.497937 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:45.497940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:45.498563 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:45.498776 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.498783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.499586 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.499587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.500445 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:45.500448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.500871 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.500873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:45.500999 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.501392 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:45.501399 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:45.502724 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:45.504120 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.504125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.504127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.504325 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.504511 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:45.504512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:45.504512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:45.504598 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:45.504938 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:45.504944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:45.505078 (+ 134us) client_negotiation.cc:769] Sending connection context
0929 06:28:45.505163 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 06:28:45.505294 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260929 06:28:45.505937 13886 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37579
I20260929 06:28:45.506039 13886 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:45.506246 13886 heartbeater.cc:511] Master 127.13.15.190:37579 requested a full tablet report, sending...
I20260929 06:28:45.506580 13426 ts_manager.cc:194] Registered new tserver with Master: 70003c70d07f40159a0baf2f4e89117c (127.13.15.131:46113)
I20260929 06:28:45.506906 13426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:57081
I20260929 06:28:45.510823 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:45.533885 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.524837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40788 (local address 127.13.15.129:37515)
0929 06:28:45.525035 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.525040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.525050 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:28:45.525077 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.525081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.525099 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.525174 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.525727 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.526493 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.527241 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.527547 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.527637 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.530426 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.530435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.530436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.530442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.532487 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.532934 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.532938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.532938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.533016 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.533251 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.533252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.533252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.533448 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.533613 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.533691 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.533768 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":121,"threads_started":1}
W20260929 06:28:45.534408 13532 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40788
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:45.554896 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:45.548661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.13.15.129:37515)
0929 06:28:45.548720 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:45.548722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:45.548794 (+ 72us) server_negotiation.cc:407] Connection header received
0929 06:28:45.548862 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:45.548863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:45.548873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:45.548945 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:45.549413 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.549858 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.550615 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:45.550801 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:45.550853 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:45.552734 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:45.552735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:45.552735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:45.552740 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:45.554078 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.554385 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.554385 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.554386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.554432 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:45.554568 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:45.554568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:45.554569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:45.554638 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:45.554713 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:45.554793 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 06:28:45.554843 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:28:45.555371 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 13489
I20260929 06:28:45.559597 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 13623
I20260929 06:28:45.564286 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 13757
I20260929 06:28:45.568292 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 13394
2026-09-29T06:28:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1858 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13900](info): set up 2 sockets
Sep 29 06:28:45 dist-test-slave-sm0p krb5kdc[13900](info): commencing operation
krb5kdc: starting...
W20260929 06:28:46.569432 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:46 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:46.677384 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43117
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:43117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:46.753011 13913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:46.753242 13913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:46.753280 13913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:46.754446 13913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:46.754503 13913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:46.754525 13913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:46.754546 13913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:46.754575 13913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:46.755908 13913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:43117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:46.756150 13913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:46.756349 13913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:28:46.760897 13913 server_base.cc:1034] running on GCE node
W20260929 06:28:46.760864 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:46.760844 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:46.760859 13918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:46.761308 13913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:46.761610 13913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:46.762761 13913 hybrid_clock.cc:648] HybridClock initialized: now 1790663326762747 us; error 26 us; skew 500 ppm
Sep 29 06:28:46 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:46.769910 13913 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:46.770346 13913 webserver.cc:460] Webserver started at http://127.13.15.190:37839/ using document root <none> and password file <none>
I20260929 06:28:46.770560 13913 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:46.770620 13913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:46.770759 13913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:46.771778 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "b6eab428f6254929ada0903f5f307660"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "be929f9f7485c3bc8fb98e27003d1ff3"
server_key_iv: "cdbff48c5fe8375e2a97ef555dc3ed9a"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.772104 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "b6eab428f6254929ada0903f5f307660"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "be929f9f7485c3bc8fb98e27003d1ff3"
server_key_iv: "cdbff48c5fe8375e2a97ef555dc3ed9a"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.773336 13913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:46.773970 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:46.774214 13913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:46.774304 13913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "b6eab428f6254929ada0903f5f307660"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "be929f9f7485c3bc8fb98e27003d1ff3"
server_key_iv: "cdbff48c5fe8375e2a97ef555dc3ed9a"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.774384 13913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:46.783349 13913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:46.784991 13913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:46.785132 13913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:46.788959 13913 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:43117
I20260929 06:28:46.788988 13980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:43117 every 8 connection(s)
I20260929 06:28:46.789316 13913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:46.790251 13981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:46.791656 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 13913
I20260929 06:28:46.791731 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:46.791882 13374 external_mini_cluster.cc:1428] Setting key 94b8b5b55eafe996a593a40d2a1735d9
I20260929 06:28:46.792781 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: Bootstrap starting.
I20260929 06:28:46.793463 13981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:46.793769 13981 log.cc:824] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:46.794510 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: No bootstrap required, opened a new log
I20260929 06:28:46.796684 13981 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } }
I20260929 06:28:46.796849 13981 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:46.796886 13981 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6eab428f6254929ada0903f5f307660, State: Initialized, Role: FOLLOWER
I20260929 06:28:46.797268 13981 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } }
I20260929 06:28:46.797363 13981 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:46.797386 13981 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:46.797457 13981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:46.797952 13981 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } }
I20260929 06:28:46.798031 13981 leader_election.cc:302] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6eab428f6254929ada0903f5f307660; no voters:
I20260929 06:28:46.798246 13981 leader_election.cc:288] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:46.798259 13986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:46.798389 13986 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [term 1 LEADER]: Becoming Leader. State: Replica: b6eab428f6254929ada0903f5f307660, State: Running, Role: LEADER
I20260929 06:28:46.798465 13986 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } }
I20260929 06:28:46.798513 13981 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:46.798806 13986 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6eab428f6254929ada0903f5f307660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } } }
I20260929 06:28:46.798883 13986 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [sys.catalog]: This master's current role is: LEADER
Sep 29 06:28:46 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:46.799005 13986 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6eab428f6254929ada0903f5f307660. Latest consensus state: current_term: 1 leader_uuid: "b6eab428f6254929ada0903f5f307660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6eab428f6254929ada0903f5f307660" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43117 } } }
I20260929 06:28:46.799069 13986 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660 [sys.catalog]: This master's current role is: LEADER
W20260929 06:28:46.800611 14002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:28:46.800668 14002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:28:46.800717 13991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:46.800848 13991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:28:46.802394 13991 catalog_manager.cc:1347] Generated new cluster ID: 03a08a465bf34f19b8b6191c637281a4
I20260929 06:28:46.802446 13991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:28:46.803124 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:46.792396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.13.15.190:43117)
0929 06:28:46.792564 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:46.792573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:46.792602 (+ 29us) server_negotiation.cc:407] Connection header received
0929 06:28:46.793143 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:46.793152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:46.793226 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:46.793362 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:46.793967 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.795336 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.796060 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.796750 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.796867 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:46.799417 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:46.799421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:46.799422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:46.799432 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:46.801371 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:46.801750 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:46.801755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:46.801755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:46.801863 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:46.802093 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:46.802094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:46.802094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:46.802458 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:46.802598 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:46.802835 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 06:28:46.802973 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260929 06:28:46.810454 13991 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:46.810940 13991 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:46.816725 13991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6eab428f6254929ada0903f5f307660: Generated new TSK 0
I20260929 06:28:46.816896 13991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:28:46.840873 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:46.918781 14007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:46.918977 14007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:46.918998 14007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:46.920153 14007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:46.920200 14007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:46.920228 14007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:28:46.921689 14007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:46.921909 14007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:46.922152 14007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:46.927098 14015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:46.927102 14013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:46.927111 14012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:46.927156 14007 server_base.cc:1034] running on GCE node
I20260929 06:28:46.927452 14007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:46.927742 14007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:46.928920 14007 hybrid_clock.cc:648] HybridClock initialized: now 1790663326928915 us; error 34 us; skew 500 ppm
Sep 29 06:28:46 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:46.931385 14007 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:28:46.931823 14007 webserver.cc:460] Webserver started at http://127.13.15.129:37109/ using document root <none> and password file <none>
I20260929 06:28:46.932035 14007 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:46.932101 14007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:46.932256 14007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:46.933257 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "cd0446bd729b41fab33e04b9861895f1"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "244a0774304e2944d005b8ad9d4a3f48"
server_key_iv: "dd191edbbf6b9f8ff8a69fea955c0dab"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.933605 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "cd0446bd729b41fab33e04b9861895f1"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "244a0774304e2944d005b8ad9d4a3f48"
server_key_iv: "dd191edbbf6b9f8ff8a69fea955c0dab"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.934969 14007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:46.935652 14022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:46.935837 14007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:46.935932 14007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "cd0446bd729b41fab33e04b9861895f1"
format_stamp: "Formatted at 2026-09-29 06:28:46 on dist-test-slave-sm0p"
server_key: "244a0774304e2944d005b8ad9d4a3f48"
server_key_iv: "dd191edbbf6b9f8ff8a69fea955c0dab"
server_key_version: "encryptionkey@0"
I20260929 06:28:46.936005 14007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:46.952476 14007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:46.954741 14007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:46.954895 14007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:46.955106 14007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:46.955396 14007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:46.955446 14007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:46.955482 14007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:46.955507 14007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:46.960996 14007 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:43837
I20260929 06:28:46.961032 14135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:43837 every 8 connection(s)
I20260929 06:28:46.961330 14007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:28:46.965211 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14007
I20260929 06:28:46.965322 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:28:46.965464 13374 external_mini_cluster.cc:1428] Setting key 0e602d5e1a64036efa2f9287b7601562
Sep 29 06:28:46 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:46.970605 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:46.962078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:36211 (local address 127.13.15.190:43117)
0929 06:28:46.962124 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:46.962126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:46.962321 (+ 195us) server_negotiation.cc:407] Connection header received
0929 06:28:46.963198 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:46.963199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:46.963208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:46.963269 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:46.963817 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.964303 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.965228 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.965578 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.965657 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:46.967875 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:46.967876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:46.967877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:46.967884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:46.969286 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:46.969644 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:46.969645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:46.969645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:46.969706 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:46.969934 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:46.969935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:46.969935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:46.970042 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:46.970113 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:46.970410 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 06:28:46.970497 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:28:46.970763 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:46.962023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43117 (local address 127.13.15.129:36211)
0929 06:28:46.962185 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:46.962231 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:46.963113 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:46.963341 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:46.963344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:46.963569 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:46.963746 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:46.963750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.964373 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.964374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:46.965137 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:46.965139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:46.965668 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:46.965669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:46.965803 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:46.966289 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:46.966299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:46.967788 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:46.969368 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:46.969373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:46.969375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:46.969563 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:46.969769 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:46.969770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:46.969771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:46.969869 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:46.970104 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:46.970109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:46.970261 (+ 152us) client_negotiation.cc:769] Sending connection context
0929 06:28:46.970388 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 06:28:46.970529 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260929 06:28:46.971190 14136 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43117
I20260929 06:28:46.971261 14136 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:46.971436 14136 heartbeater.cc:511] Master 127.13.15.190:43117 requested a full tablet report, sending...
I20260929 06:28:46.971992 13945 ts_manager.cc:194] Registered new tserver with Master: cd0446bd729b41fab33e04b9861895f1 (127.13.15.129:43837)
I20260929 06:28:46.972891 13945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:36211
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:28:46.990051 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:47.066583 14141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:47.066782 14141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:47.066803 14141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:47.067854 14141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:47.067885 14141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:47.067909 14141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:28:47.069321 14141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:47.069482 14141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:47.069641 14141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:47.074471 14147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:47.074482 14146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:47.074472 14149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:47.074566 14141 server_base.cc:1034] running on GCE node
I20260929 06:28:47.074831 14141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:47.075071 14141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:47.076242 14141 hybrid_clock.cc:648] HybridClock initialized: now 1790663327076207 us; error 46 us; skew 500 ppm
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:47.078717 14141 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:28:47.079176 14141 webserver.cc:460] Webserver started at http://127.13.15.130:34265/ using document root <none> and password file <none>
I20260929 06:28:47.079389 14141 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:47.079452 14141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:47.079578 14141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:47.080541 14141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "064950b9b12b73f1dfbfe54ade5f1e61"
server_key_iv: "378362ba0baf44613ab9980bf850348a"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.080888 14141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "064950b9b12b73f1dfbfe54ade5f1e61"
server_key_iv: "378362ba0baf44613ab9980bf850348a"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.082221 14141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:28:47.082906 14156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.083104 14141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:47.083206 14141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "064950b9b12b73f1dfbfe54ade5f1e61"
server_key_iv: "378362ba0baf44613ab9980bf850348a"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.083287 14141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:47.097756 14141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:47.099712 14141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:47.099838 14141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:47.100041 14141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:47.100328 14141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:47.100385 14141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.100423 14141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:47.100449 14141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.105867 14141 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:40303
I20260929 06:28:47.105938 14269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:40303 every 8 connection(s)
I20260929 06:28:47.106305 14141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:47.114214 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14141
I20260929 06:28:47.114317 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:28:47.114467 13374 external_mini_cluster.cc:1428] Setting key 2c637a939b0159dbf595cf60f475344b
I20260929 06:28:47.115532 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.107015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:34439 (local address 127.13.15.190:43117)
0929 06:28:47.107081 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.107084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.107281 (+ 197us) server_negotiation.cc:407] Connection header received
0929 06:28:47.108169 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.108171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.108182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.108247 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.108834 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.109331 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.110335 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.110522 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.110593 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.112506 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.112507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.112508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.112515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.113936 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.114441 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.114442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.114443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.114499 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.114776 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.114777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.114777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.114922 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.114985 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.115293 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.115389 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 06:28:47.115542 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.107005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43117 (local address 127.13.15.130:34439)
0929 06:28:47.107146 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.107207 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.108062 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.108315 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.108319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.108561 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:47.108741 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.108745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.109437 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.109438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.110225 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.110228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.110598 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.110599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.110719 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.111184 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:47.111191 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:47.112420 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:47.114034 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:47.114040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:47.114042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:47.114370 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:47.114589 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:47.114590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:47.114590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:47.114696 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:47.114988 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:47.114993 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:47.115158 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.115264 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.115380 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260929 06:28:47.116055 14270 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43117
I20260929 06:28:47.116148 14270 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:47.116330 14270 heartbeater.cc:511] Master 127.13.15.190:43117 requested a full tablet report, sending...
I20260929 06:28:47.116742 13945 ts_manager.cc:194] Registered new tserver with Master: ad605e07e6fc4d45ad60373d6dbb5a5f (127.13.15.130:40303)
I20260929 06:28:47.117235 13945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:34439
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:28:47.138098 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:47.215907 14275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:47.216099 14275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:47.216120 14275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:47.217296 14275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:47.217351 14275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:47.217376 14275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:28:47.218905 14275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:47.219092 14275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:47.219300 14275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:47.223870 14281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:47.223865 14283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:47.223954 14280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:47.224089 14275 server_base.cc:1034] running on GCE node
I20260929 06:28:47.224298 14275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:47.224469 14275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:47.225596 14275 hybrid_clock.cc:648] HybridClock initialized: now 1790663327225576 us; error 42 us; skew 500 ppm
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:47.228018 14275 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:28:47.228407 14275 webserver.cc:460] Webserver started at http://127.13.15.131:41839/ using document root <none> and password file <none>
I20260929 06:28:47.228626 14275 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:47.228696 14275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:47.228838 14275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:47.229765 14275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "70341c8c25394284b49222ba5b7331e4"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "a2c9a9d4d271aeb2b047a3242b62ade4"
server_key_iv: "10fd8f2454a89a561e57e82f2f574df6"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.230108 14275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "70341c8c25394284b49222ba5b7331e4"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "a2c9a9d4d271aeb2b047a3242b62ade4"
server_key_iv: "10fd8f2454a89a561e57e82f2f574df6"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.231403 14275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:47.232089 14290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.232270 14275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:47.232352 14275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "70341c8c25394284b49222ba5b7331e4"
format_stamp: "Formatted at 2026-09-29 06:28:47 on dist-test-slave-sm0p"
server_key: "a2c9a9d4d271aeb2b047a3242b62ade4"
server_key_iv: "10fd8f2454a89a561e57e82f2f574df6"
server_key_version: "encryptionkey@0"
I20260929 06:28:47.232425 14275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:47.257006 14275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:47.258620 14275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:47.258769 14275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:47.258970 14275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:47.259253 14275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:47.259301 14275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.259354 14275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:47.259380 14275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:47.264714 14275 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:43051
I20260929 06:28:47.264787 14403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:43051 every 8 connection(s)
I20260929 06:28:47.265084 14275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:47.272452 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14275
I20260929 06:28:47.272555 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:28:47.272725 13374 external_mini_cluster.cc:1428] Setting key 88e383fef85b84989a6d890e014887ce
I20260929 06:28:47.274323 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.265882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:42809 (local address 127.13.15.190:43117)
0929 06:28:47.265939 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.265941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.266268 (+ 327us) server_negotiation.cc:407] Connection header received
0929 06:28:47.267103 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.267104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.267113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.267165 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.267697 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.268163 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.269113 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.269300 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.269347 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.271420 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.271421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.271422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.271428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.272908 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.273395 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.273396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.273396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.273468 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.273728 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.273729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.273729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.273829 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.273882 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.274114 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.274251 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:28:47.274493 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.265858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43117 (local address 127.13.15.131:42809)
0929 06:28:47.266007 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.266045 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.267018 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.267224 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.267227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.267459 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:47.267627 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.267631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.268241 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.268241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.269044 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.269046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.269364 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.269364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.269454 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.269942 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:47.269951 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:47.271340 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:47.273009 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:47.273028 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:47.273031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:47.273329 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:47.273538 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:47.273539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:47.273539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:47.273671 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:47.273886 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:47.273891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:47.274008 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.274085 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.274276 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260929 06:28:47.274923 14404 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43117
I20260929 06:28:47.275013 14404 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:47.275193 14404 heartbeater.cc:511] Master 127.13.15.190:43117 requested a full tablet report, sending...
I20260929 06:28:47.275545 13945 ts_manager.cc:194] Registered new tserver with Master: 70341c8c25394284b49222ba5b7331e4 (127.13.15.131:43051)
I20260929 06:28:47.275889 13945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:42809
I20260929 06:28:47.284296 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:28:47.291342 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.285740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.13.15.190:43117)
0929 06:28:47.285821 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.285822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.285978 (+ 156us) server_negotiation.cc:407] Connection header received
0929 06:28:47.285999 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.286000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.286010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.286054 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.286689 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.287194 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.287947 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.288137 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.288219 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.289218 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.289220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.289220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.289226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.290221 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.290568 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.290569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.290569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.290619 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.290829 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.290830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.290831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.290967 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.291073 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.291177 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.291249 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 06:28:47.293192 13945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:28:47.294759 13945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:47.301093 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.296804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:43837 (local address 127.0.0.1:55514)
0929 06:28:47.297077 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.297085 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.297124 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.297527 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.297528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.297531 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:47.297585 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.297587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.299055 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.299056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.300017 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.300020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300781 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.300858 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.300871 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.300937 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.300996 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":61,"threads_started":1}
I20260929 06:28:47.301148 14417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.297123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.13.15.130:40303)
0929 06:28:47.297275 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.297278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.297286 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:47.297313 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.297315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.297326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.297412 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:47.297531 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.298434 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300158 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300818 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300882 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.300927 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.301016 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.301073 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260929 06:28:47.301226 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.296799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:43051 (local address 127.0.0.1:45276)
0929 06:28:47.296968 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.297004 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.297062 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.297584 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.297585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.297587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:47.297617 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.297619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.298908 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.298909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.300063 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.300065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300960 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.301028 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.301032 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.301098 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.301135 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":92,"thread_start_us":58,"threads_started":1}
I20260929 06:28:47.301388 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.296865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.13.15.131:43051)
0929 06:28:47.297334 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.297339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.297350 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:28:47.297392 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.297398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.297412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.297497 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:47.297697 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.298793 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300158 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300818 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300881 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.300927 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.301282 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.301324 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":94,"threads_started":1}
I20260929 06:28:47.301534 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.297009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:40303 (local address 127.0.0.1:34892)
0929 06:28:47.297125 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.297136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.297189 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.297415 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.297418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.297420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:47.297475 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.297477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.298555 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.298556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.300072 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.300074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300884 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.300939 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.300942 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.301000 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.301037 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":51,"thread_start_us":65,"threads_started":1}
I20260929 06:28:47.301816 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.296899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.13.15.129:43837)
0929 06:28:47.297334 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.297339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.297350 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:28:47.297392 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.297394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.297411 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.297688 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:47.297769 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.298916 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300112 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.300683 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.300777 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.300831 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.301694 (+ 863us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.301740 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":103,"threads_started":1}
I20260929 06:28:47.302030 14338 tablet_service.cc:1467] Processing CreateTablet for tablet a4b8131658f04e2ebe6acb34a1323031 (DEFAULT_TABLE table=test-table [id=f70311cc28dd4b54999c473e6b9e6a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:47.302241 14204 tablet_service.cc:1467] Processing CreateTablet for tablet a4b8131658f04e2ebe6acb34a1323031 (DEFAULT_TABLE table=test-table [id=f70311cc28dd4b54999c473e6b9e6a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:47.302402 14338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b8131658f04e2ebe6acb34a1323031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:47.302485 14204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b8131658f04e2ebe6acb34a1323031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:47.302445 14070 tablet_service.cc:1467] Processing CreateTablet for tablet a4b8131658f04e2ebe6acb34a1323031 (DEFAULT_TABLE table=test-table [id=f70311cc28dd4b54999c473e6b9e6a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:47.302722 14070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b8131658f04e2ebe6acb34a1323031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:47.304728 14418 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Bootstrap starting.
I20260929 06:28:47.305380 14418 tablet_bootstrap.cc:654] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:47.305637 14419 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: Bootstrap starting.
I20260929 06:28:47.305968 14418 log.cc:824] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:47.306025 14420 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: Bootstrap starting.
I20260929 06:28:47.306420 14419 tablet_bootstrap.cc:654] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:47.306722 14420 tablet_bootstrap.cc:654] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:47.306892 14418 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: No bootstrap required, opened a new log
I20260929 06:28:47.306960 14418 ts_tablet_manager.cc:1397] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:28:47.307010 14419 log.cc:824] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:47.307320 14420 log.cc:824] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:47.307860 14419 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: No bootstrap required, opened a new log
I20260929 06:28:47.307926 14419 ts_tablet_manager.cc:1397] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:28:47.308032 14420 tablet_bootstrap.cc:492] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: No bootstrap required, opened a new log
I20260929 06:28:47.308123 14420 ts_tablet_manager.cc:1397] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:28:47.308876 14418 raft_consensus.cc:348] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.309042 14418 raft_consensus.cc:374] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:47.309077 14418 raft_consensus.cc:729] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad605e07e6fc4d45ad60373d6dbb5a5f, State: Initialized, Role: FOLLOWER
I20260929 06:28:47.309219 14418 consensus_queue.cc:260] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.309485 14418 ts_tablet_manager.cc:1428] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 06:28:47.309621 14270 heartbeater.cc:503] Master 127.13.15.190:43117 was elected leader, sending a full tablet report...
I20260929 06:28:47.309755 14420 raft_consensus.cc:348] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.309809 14419 raft_consensus.cc:348] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.309932 14420 raft_consensus.cc:374] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:47.309978 14420 raft_consensus.cc:729] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0446bd729b41fab33e04b9861895f1, State: Initialized, Role: FOLLOWER
I20260929 06:28:47.309979 14419 raft_consensus.cc:374] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:47.310021 14419 raft_consensus.cc:729] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70341c8c25394284b49222ba5b7331e4, State: Initialized, Role: FOLLOWER
I20260929 06:28:47.310112 14420 consensus_queue.cc:260] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.310156 14419 consensus_queue.cc:260] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.310415 14419 ts_tablet_manager.cc:1428] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:28:47.310418 14136 heartbeater.cc:503] Master 127.13.15.190:43117 was elected leader, sending a full tablet report...
I20260929 06:28:47.310554 14404 heartbeater.cc:503] Master 127.13.15.190:43117 was elected leader, sending a full tablet report...
I20260929 06:28:47.310420 14420 ts_tablet_manager.cc:1428] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:28:47.350543 14425 raft_consensus.cc:482] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:47.350652 14425 raft_consensus.cc:504] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.350957 14425 leader_election.cc:288] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70341c8c25394284b49222ba5b7331e4 (127.13.15.131:43051), ad605e07e6fc4d45ad60373d6dbb5a5f (127.13.15.130:40303)
I20260929 06:28:47.354807 14417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.351309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:51695 (local address 127.13.15.130:40303)
0929 06:28:47.351394 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.351396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.351469 (+ 73us) server_negotiation.cc:407] Connection header received
0929 06:28:47.351533 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.351534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.351542 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.351589 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:47.351667 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.352314 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.353843 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.354437 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.354508 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.354516 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.354711 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.354746 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 06:28:47.354807 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.351156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:58909 (local address 127.13.15.131:43051)
0929 06:28:47.351211 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.351212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.351248 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:28:47.351330 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.351331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.351341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.351398 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:47.351542 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.352260 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.353843 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.354437 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.354508 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.354516 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.354711 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.354746 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:28:47.354837 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.351129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:43051 (local address 127.13.15.129:58909)
0929 06:28:47.351197 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.351210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.351265 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.351411 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.351412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.351414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:47.351479 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.351480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.352387 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.352389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.353750 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.353753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.354561 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.354562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.354640 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.354643 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.354702 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.354730 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":86}
I20260929 06:28:47.355065 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.351254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:40303 (local address 127.13.15.129:51695)
0929 06:28:47.351403 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:47.351412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:47.351463 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:47.351589 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:47.351590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:47.351591 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:47.351630 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.351631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.352396 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.352397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.353750 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:47.353753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.354534 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.354535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:47.354609 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.354614 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:28:47.354684 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 06:28:47.354727 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":77,"thread_start_us":93,"threads_started":1}
I20260929 06:28:47.355152 14358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b8131658f04e2ebe6acb34a1323031" candidate_uuid: "cd0446bd729b41fab33e04b9861895f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70341c8c25394284b49222ba5b7331e4" is_pre_election: true
I20260929 06:28:47.355314 14358 raft_consensus.cc:2454] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0446bd729b41fab33e04b9861895f1 in term 0.
I20260929 06:28:47.355434 14224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b8131658f04e2ebe6acb34a1323031" candidate_uuid: "cd0446bd729b41fab33e04b9861895f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" is_pre_election: true
I20260929 06:28:47.355502 14023 leader_election.cc:302] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70341c8c25394284b49222ba5b7331e4, cd0446bd729b41fab33e04b9861895f1; no voters:
I20260929 06:28:47.355587 14224 raft_consensus.cc:2454] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0446bd729b41fab33e04b9861895f1 in term 0.
I20260929 06:28:47.355628 14425 raft_consensus.cc:2781] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:47.355690 14425 raft_consensus.cc:482] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:47.355717 14425 raft_consensus.cc:3037] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:47.356462 14425 raft_consensus.cc:504] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.356614 14425 leader_election.cc:288] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [CANDIDATE]: Term 1 election: Requested vote from peers 70341c8c25394284b49222ba5b7331e4 (127.13.15.131:43051), ad605e07e6fc4d45ad60373d6dbb5a5f (127.13.15.130:40303)
I20260929 06:28:47.356782 14358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b8131658f04e2ebe6acb34a1323031" candidate_uuid: "cd0446bd729b41fab33e04b9861895f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70341c8c25394284b49222ba5b7331e4"
I20260929 06:28:47.356808 14224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b8131658f04e2ebe6acb34a1323031" candidate_uuid: "cd0446bd729b41fab33e04b9861895f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f"
W20260929 06:28:47.356828 14271 tablet.cc:2369] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:28:47.356866 14358 raft_consensus.cc:3037] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:47.356873 14224 raft_consensus.cc:3037] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:47.357774 14358 raft_consensus.cc:2454] T a4b8131658f04e2ebe6acb34a1323031 P 70341c8c25394284b49222ba5b7331e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0446bd729b41fab33e04b9861895f1 in term 1.
I20260929 06:28:47.357774 14224 raft_consensus.cc:2454] T a4b8131658f04e2ebe6acb34a1323031 P ad605e07e6fc4d45ad60373d6dbb5a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0446bd729b41fab33e04b9861895f1 in term 1.
I20260929 06:28:47.357949 14023 leader_election.cc:302] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad605e07e6fc4d45ad60373d6dbb5a5f, cd0446bd729b41fab33e04b9861895f1; no voters:
I20260929 06:28:47.358039 14425 raft_consensus.cc:2781] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:47.358219 14425 raft_consensus.cc:686] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [term 1 LEADER]: Becoming Leader. State: Replica: cd0446bd729b41fab33e04b9861895f1, State: Running, Role: LEADER
I20260929 06:28:47.358340 14425 consensus_queue.cc:237] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } }
I20260929 06:28:47.359187 13944 catalog_manager.cc:5495] T a4b8131658f04e2ebe6acb34a1323031 P cd0446bd729b41fab33e04b9861895f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd0446bd729b41fab33e04b9861895f1 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "cd0446bd729b41fab33e04b9861895f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70341c8c25394284b49222ba5b7331e4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad605e07e6fc4d45ad60373d6dbb5a5f" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 40303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd0446bd729b41fab33e04b9861895f1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43837 } health_report { overall_health: HEALTHY } } }
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:47.379611 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.372083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.13.15.129:43837)
0929 06:28:47.372156 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.372158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.372273 (+ 115us) server_negotiation.cc:407] Connection header received
0929 06:28:47.372348 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.372349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.372360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.372409 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.372923 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.373382 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.374088 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.374337 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.374415 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.376736 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.376744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.376744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.376751 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.378656 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.378953 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.378956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.378956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.379032 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.379185 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.379186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.379186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.379313 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.379421 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.379454 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.379550 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:47.401916 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.395065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.13.15.129:43837)
0929 06:28:47.395145 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.395147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.395249 (+ 102us) server_negotiation.cc:407] Connection header received
0929 06:28:47.395328 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.395329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.395339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.395419 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.395947 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.396417 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.397144 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.397355 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.397428 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.399679 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.399680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.399681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.399687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.401035 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.401320 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.401320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.401320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.401368 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.401506 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.401506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.401506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.401581 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.401647 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.401761 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.401817 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[13900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:47.422300 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:47.416240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.13.15.129:43837)
0929 06:28:47.416301 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:47.416303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:47.416382 (+ 79us) server_negotiation.cc:407] Connection header received
0929 06:28:47.416420 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:47.416420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:47.416429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:47.416475 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:47.416948 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.417384 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.418032 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:47.418262 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:47.418312 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:47.420237 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:47.420239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:47.420239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:47.420245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:47.421387 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.421678 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.421679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.421679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.421724 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:47.421873 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:47.421873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:47.421874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:47.421953 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:47.422059 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:47.422129 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 06:28:47.422240 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260929 06:28:47.422757 14050 tablet_service.cc:2908] Rejecting scan request for tablet a4b8131658f04e2ebe6acb34a1323031: Uninitialized: clean time has not yet been initialized
I20260929 06:28:47.423501 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14007
I20260929 06:28:47.428565 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14141
I20260929 06:28:47.433167 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14275
I20260929 06:28:47.437192 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 13913
2026-09-29T06:28:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1870 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[14442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[14442](info): set up 2 sockets
Sep 29 06:28:47 dist-test-slave-sm0p krb5kdc[14442](info): commencing operation
krb5kdc: starting...
W20260929 06:28:48.449366 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:48.564220 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35971
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:35971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:48.642938 14455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:48.643122 14455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:48.643144 14455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:48.644196 14455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:48.644229 14455 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 06:28:48.644241 14455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:48.644254 14455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:48.644266 14455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:48.644279 14455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:48.645684 14455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:35971
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:48.645852 14455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:48.646008 14455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:48.650781 14461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:48.650794 14460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:48.650808 14455 server_base.cc:1034] running on GCE node
W20260929 06:28:48.650794 14463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:48.651182 14455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:48.651436 14455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:48.652626 14455 hybrid_clock.cc:648] HybridClock initialized: now 1790663328652609 us; error 30 us; skew 500 ppm
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:48.655117 14455 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:48.655552 14455 webserver.cc:460] Webserver started at http://127.13.15.190:33269/ using document root <none> and password file <none>
I20260929 06:28:48.655787 14455 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:48.655830 14455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:48.655977 14455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:48.656986 14455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "31e1eec403aa44299ede41e63e26bf39"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "c7ee9bcf7d0c406e32054411a587a9d3"
server_key_iv: "f17c4201331eb3f0d5b7360c058b9a76"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.657325 14455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "31e1eec403aa44299ede41e63e26bf39"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "c7ee9bcf7d0c406e32054411a587a9d3"
server_key_iv: "f17c4201331eb3f0d5b7360c058b9a76"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.658689 14455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:48.659350 14470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:48.659521 14455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:48.659616 14455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "31e1eec403aa44299ede41e63e26bf39"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "c7ee9bcf7d0c406e32054411a587a9d3"
server_key_iv: "f17c4201331eb3f0d5b7360c058b9a76"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.659710 14455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:48.704733 14455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:48.706768 14455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:48.706951 14455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:48.711799 14455 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:35971
I20260929 06:28:48.711858 14532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:35971 every 8 connection(s)
I20260929 06:28:48.712159 14455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:48.712750 14533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:48.715054 14533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Bootstrap starting.
I20260929 06:28:48.715569 14533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:48.715830 14533 log.cc:824] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:48.716468 14533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: No bootstrap required, opened a new log
I20260929 06:28:48.718537 14533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } }
I20260929 06:28:48.718707 14533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:48.718747 14533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e1eec403aa44299ede41e63e26bf39, State: Initialized, Role: FOLLOWER
I20260929 06:28:48.718914 14533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } }
I20260929 06:28:48.718950 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14455
I20260929 06:28:48.719005 14533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:48.719048 14533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:48.719039 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:48.719096 14533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:48.719238 13374 external_mini_cluster.cc:1428] Setting key edc4b1e557266a44182f6e3b8fad83f9
I20260929 06:28:48.719795 14533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } }
I20260929 06:28:48.719913 14533 leader_election.cc:302] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e1eec403aa44299ede41e63e26bf39; no voters:
I20260929 06:28:48.720054 14533 leader_election.cc:288] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:48.720130 14538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:48.720278 14538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [term 1 LEADER]: Becoming Leader. State: Replica: 31e1eec403aa44299ede41e63e26bf39, State: Running, Role: LEADER
I20260929 06:28:48.720386 14538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } }
I20260929 06:28:48.720278 14533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:48.720857 14540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e1eec403aa44299ede41e63e26bf39. Latest consensus state: current_term: 1 leader_uuid: "31e1eec403aa44299ede41e63e26bf39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } } }
I20260929 06:28:48.720938 14540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:48.720841 14539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e1eec403aa44299ede41e63e26bf39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e1eec403aa44299ede41e63e26bf39" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35971 } } }
I20260929 06:28:48.721055 14539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:48.721156 14544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:48.721691 14544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:28:48.723688 14544 catalog_manager.cc:1347] Generated new cluster ID: 735d07022bae4d369492d138fbf29734
I20260929 06:28:48.723763 14544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:48.730823 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:48.719710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.13.15.190:35971)
0929 06:28:48.719903 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:48.719912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:48.719945 (+ 33us) server_negotiation.cc:407] Connection header received
0929 06:28:48.720059 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:48.720066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:48.720123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:48.721008 (+ 885us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:48.721122 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.722576 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.723632 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.724268 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.724386 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:48.727198 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:48.727202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:48.727202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:48.727213 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:48.729190 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.729585 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.729587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.729588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.729703 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.729923 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.729923 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.729924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.730226 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:48.730411 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:48.730503 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 06:28:48.730627 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260929 06:28:48.733033 14544 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:48.733521 14544 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:48.737094 14544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Generated new TSK 0
I20260929 06:28:48.737253 14544 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:28:48.761104 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:35971
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:48.838181 14559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:48.838364 14559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:48.838387 14559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:48.839520 14559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:48.839560 14559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:48.839581 14559 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:48.839596 14559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:28:48.840965 14559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:35971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:48.841140 14559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:48.841382 14559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:48.846175 14564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:48.846226 14567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:48.846247 14565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:48.846232 14559 server_base.cc:1034] running on GCE node
I20260929 06:28:48.846575 14559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:48.846841 14559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:48.848012 14559 hybrid_clock.cc:648] HybridClock initialized: now 1790663328848005 us; error 33 us; skew 500 ppm
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:48.850380 14559 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:28:48.850796 14559 webserver.cc:460] Webserver started at http://127.13.15.129:44829/ using document root <none> and password file <none>
I20260929 06:28:48.851029 14559 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:48.851096 14559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:48.851251 14559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:48.852237 14559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "a816833c1e75457798af1dd72afd84a6"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "a288d12b474b61c760d04a6e321fad04"
server_key_iv: "0a7644d2b91e31935d0eb3a4f027af54"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.852576 14559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "a816833c1e75457798af1dd72afd84a6"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "a288d12b474b61c760d04a6e321fad04"
server_key_iv: "0a7644d2b91e31935d0eb3a4f027af54"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.853812 14559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:48.854568 14574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:48.854739 14559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:48.854848 14559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "a816833c1e75457798af1dd72afd84a6"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "a288d12b474b61c760d04a6e321fad04"
server_key_iv: "0a7644d2b91e31935d0eb3a4f027af54"
server_key_version: "encryptionkey@0"
I20260929 06:28:48.854928 14559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:48.866693 14559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:48.868352 14559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:48.868500 14559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:48.868996 14559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:48.869064 14559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:48.869103 14559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:48.869129 14559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:48.875845 14559 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:44111
I20260929 06:28:48.875917 14689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:44111 every 8 connection(s)
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:48.876241 14559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:28:48.878858 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:48.869471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53199 (local address 127.13.15.190:35971)
0929 06:28:48.869524 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:48.869526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:48.869695 (+ 169us) server_negotiation.cc:407] Connection header received
0929 06:28:48.870214 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:48.870215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:48.870227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:48.870294 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:48.871087 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.871886 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.873534 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.873866 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.873935 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:48.876600 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:48.876601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:48.876602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:48.876612 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:48.877712 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.878077 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.878078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.878078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.878125 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.878369 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.878369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.878369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.878467 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:48.878516 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:48.878731 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 06:28:48.878794 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 06:28:48.878999 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:48.869444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.129:53199)
0929 06:28:48.869591 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:48.869628 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:48.870080 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:48.870362 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:48.870366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:48.870695 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:48.870964 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:48.870969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.871994 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.871995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:48.873422 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:48.873425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.873953 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.873954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:48.874113 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:48.874718 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:48.874729 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:48.876493 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:48.877784 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:48.877789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:48.877790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:48.878011 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:48.878215 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:48.878216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:48.878216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:48.878313 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:48.878522 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:48.878526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:48.878625 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 06:28:48.878710 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 06:28:48.878827 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260929 06:28:48.879532 14690 heartbeater.cc:347] Connected to a master server at 127.13.15.190:35971
I20260929 06:28:48.879611 14690 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:48.879822 14690 heartbeater.cc:511] Master 127.13.15.190:35971 requested a full tablet report, sending...
I20260929 06:28:48.880330 14486 ts_manager.cc:194] Registered new tserver with Master: a816833c1e75457798af1dd72afd84a6 (127.13.15.129:44111)
I20260929 06:28:48.881134 14486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53199
I20260929 06:28:48.885147 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14559
I20260929 06:28:48.885269 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:28:48.885465 13374 external_mini_cluster.cc:1428] Setting key 88a2fb016d614bed4afa60441835872e
I20260929 06:28:48.885504 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:48.880641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53617 (local address 127.13.15.190:35971)
0929 06:28:48.880689 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:48.880691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:48.880714 (+ 23us) server_negotiation.cc:407] Connection header received
0929 06:28:48.880732 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:48.880733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:48.880741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:48.880787 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:48.881031 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.881612 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.882352 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.882551 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.882630 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:48.883408 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:48.883409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:48.883409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:48.883415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:48.884427 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.884767 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.884768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.884769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.884822 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:48.885011 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:48.885012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:48.885013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:48.885119 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:48.885193 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:48.885346 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 06:28:48.885417 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 06:28:48.885519 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:48.880450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.129:53617)
0929 06:28:48.880594 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:48.880606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:48.880670 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:48.880790 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:48.880791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:48.880923 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:48.880975 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:48.880976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.881714 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.881715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:48.882266 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:48.882267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:48.882630 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:48.882631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:48.882711 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:48.883016 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:48.883020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:48.883312 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:48.884521 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:48.884522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:48.884523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:48.884681 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:48.884889 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:48.884889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:48.884890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:48.884944 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:48.885186 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:48.885187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:48.885239 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 06:28:48.885323 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 06:28:48.885417 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:28:48.909473 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:35971
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:48.988415 14699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:48.988603 14699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:48.988626 14699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:48.989737 14699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:48.989775 14699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:48.989796 14699 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:48.989856 14699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:28:48.991401 14699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:35971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:48.991583 14699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:48.991824 14699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:48.996614 14707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:48.996666 14705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:48.996621 14704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:48.996827 14699 server_base.cc:1034] running on GCE node
I20260929 06:28:48.997010 14699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:48.997164 14699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:48.998278 14699 hybrid_clock.cc:648] HybridClock initialized: now 1790663328998260 us; error 28 us; skew 500 ppm
Sep 29 06:28:48 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:49.000510 14699 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:28:49.000936 14699 webserver.cc:460] Webserver started at http://127.13.15.130:38499/ using document root <none> and password file <none>
I20260929 06:28:49.001140 14699 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:49.001219 14699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:49.001370 14699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:49.002391 14699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "a0bfa14a41a647e18553b16f6e4e49c3"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "b54be9852fcf6f52bd544cb76c7151dd"
server_key_iv: "2dc5cd07f48c38086dc285800e1e6c2a"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.002743 14699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "a0bfa14a41a647e18553b16f6e4e49c3"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "b54be9852fcf6f52bd544cb76c7151dd"
server_key_iv: "2dc5cd07f48c38086dc285800e1e6c2a"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.004045 14699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:28:49.004712 14714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.004873 14699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:49.004964 14699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "a0bfa14a41a647e18553b16f6e4e49c3"
format_stamp: "Formatted at 2026-09-29 06:28:48 on dist-test-slave-sm0p"
server_key: "b54be9852fcf6f52bd544cb76c7151dd"
server_key_iv: "2dc5cd07f48c38086dc285800e1e6c2a"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.005040 14699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:49.016963 14699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:49.019132 14699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:49.019297 14699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:49.019776 14699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:49.019836 14699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.019869 14699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:49.019894 14699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:49.026844 14699 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:44477
I20260929 06:28:49.027179 14699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:28:49.027175 14829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:44477 every 8 connection(s)
I20260929 06:28:49.029138 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.020075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:48115 (local address 127.13.15.190:35971)
0929 06:28:49.020120 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.020122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.020300 (+ 178us) server_negotiation.cc:407] Connection header received
0929 06:28:49.020790 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.020791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.020802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.020841 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.021589 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.022391 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.023377 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.023585 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.023670 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.026161 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.026162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.026163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.026170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.027770 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.028193 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.028194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.028195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.028250 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.028510 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.028511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.028512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.028628 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.028695 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.028968 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.029057 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:28:49.029295 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.020071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.130:48115)
0929 06:28:49.020196 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.020237 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.020699 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.020898 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.020903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.021253 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:49.021507 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.021513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.022493 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.022494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.023272 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.023274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.023676 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.023677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.023811 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.024363 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:49.024371 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:49.026039 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:49.027858 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.027865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.027867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.028105 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.028322 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.028323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.028323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.028444 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.028688 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:49.028693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:49.028844 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.028946 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.029091 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260929 06:28:49.029722 14830 heartbeater.cc:347] Connected to a master server at 127.13.15.190:35971
I20260929 06:28:49.029815 14830 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:49.029968 14830 heartbeater.cc:511] Master 127.13.15.190:35971 requested a full tablet report, sending...
I20260929 06:28:49.030331 14487 ts_manager.cc:194] Registered new tserver with Master: a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477)
I20260929 06:28:49.030694 14487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:48115
I20260929 06:28:49.033943 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14699
I20260929 06:28:49.034029 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:28:49.034225 13374 external_mini_cluster.cc:1428] Setting key 9f61c3af05e54578977e669d465b7bf7
I20260929 06:28:49.035741 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.030804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:42309 (local address 127.13.15.190:35971)
0929 06:28:49.030844 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.030846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.030867 (+ 21us) server_negotiation.cc:407] Connection header received
0929 06:28:49.030929 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.030930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.030939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.030986 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.031276 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.031728 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.032375 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.032594 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.032673 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.033430 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.033431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.033432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.033438 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.034733 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.035048 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.035049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.035049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.035099 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.035291 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.035292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.035292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.035407 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.035476 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.035632 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.035688 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:28:49.035863 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.030690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.130:42309)
0929 06:28:49.030805 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.030815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.030877 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.030992 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.030992 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.031142 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:49.031194 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.031195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.031808 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.031809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.032297 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.032298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.032674 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.032675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.032742 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.033083 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:49.033086 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:49.033336 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:49.034791 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.034792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.034793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.034981 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.035164 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.035165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.035165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.035234 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.035487 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:49.035488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:49.035535 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.035606 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.035708 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:28:49.056886 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:35971
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:49.134543 14839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:49.134734 14839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:49.134758 14839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:49.135898 14839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:49.135937 14839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:49.135957 14839 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:49.136013 14839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:28:49.137347 14839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:35971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:49.137567 14839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:49.137779 14839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:49.142442 14847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:49.142452 14844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:49.142452 14845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:49.142632 14839 server_base.cc:1034] running on GCE node
I20260929 06:28:49.142881 14839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:49.143116 14839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:49.144263 14839 hybrid_clock.cc:648] HybridClock initialized: now 1790663329144247 us; error 31 us; skew 500 ppm
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:49.146699 14839 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:28:49.147099 14839 webserver.cc:460] Webserver started at http://127.13.15.131:37357/ using document root <none> and password file <none>
I20260929 06:28:49.147320 14839 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:49.147387 14839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:49.147528 14839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:49.148592 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
format_stamp: "Formatted at 2026-09-29 06:28:49 on dist-test-slave-sm0p"
server_key: "c4dfa4168ea57f2f32ae78dc8b2f738e"
server_key_iv: "a9733181ec2716082357520b91b50688"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.148939 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
format_stamp: "Formatted at 2026-09-29 06:28:49 on dist-test-slave-sm0p"
server_key: "c4dfa4168ea57f2f32ae78dc8b2f738e"
server_key_iv: "a9733181ec2716082357520b91b50688"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.150269 14839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:49.151095 14854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.151275 14839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:49.151365 14839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
format_stamp: "Formatted at 2026-09-29 06:28:49 on dist-test-slave-sm0p"
server_key: "c4dfa4168ea57f2f32ae78dc8b2f738e"
server_key_iv: "a9733181ec2716082357520b91b50688"
server_key_version: "encryptionkey@0"
I20260929 06:28:49.151449 14839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:49.166505 14839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:49.168560 14839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:49.168727 14839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:49.169179 14839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:49.169237 14839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.169310 14839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:49.169340 14839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.176234 14839 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:34149
I20260929 06:28:49.176306 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:34149 every 8 connection(s)
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:49.176594 14839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:28:49.179095 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.169759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:56063 (local address 127.13.15.190:35971)
0929 06:28:49.169844 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.169846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.170035 (+ 189us) server_negotiation.cc:407] Connection header received
0929 06:28:49.170573 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.170574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.170587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.170642 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.171473 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.172284 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.173863 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.174219 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.174300 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.176941 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.176942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.176942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.176948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.177997 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.178404 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.178405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.178405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.178453 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.178619 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.178620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.178620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.178715 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.178762 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.178979 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.179036 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 06:28:49.179225 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.169733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.131:56063)
0929 06:28:49.169908 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.169944 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.170449 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.170702 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.170706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.171083 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:49.171371 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.171377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.172405 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.172406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.173763 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.173766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.174319 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.174320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.174482 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.175031 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:49.175040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:49.176839 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:49.178064 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.178069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.178070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.178340 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.178493 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.178494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.178494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.178572 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.178770 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:49.178774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:49.178868 (+ 94us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.178942 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.179043 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260929 06:28:49.179685 14970 heartbeater.cc:347] Connected to a master server at 127.13.15.190:35971
I20260929 06:28:49.179806 14970 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:49.179965 14970 heartbeater.cc:511] Master 127.13.15.190:35971 requested a full tablet report, sending...
I20260929 06:28:49.180289 14487 ts_manager.cc:194] Registered new tserver with Master: 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.180769 14487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:56063
I20260929 06:28:49.181107 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 14839
I20260929 06:28:49.181211 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:28:49.181370 13374 external_mini_cluster.cc:1428] Setting key eef58e3ca48f5505188452f6a10559a4
I20260929 06:28:49.182024 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:28:49.186347 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.180700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:37371 (local address 127.13.15.190:35971)
0929 06:28:49.180886 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.180888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.180895 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:28:49.180912 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.180913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.180922 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.180962 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.181335 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.182000 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.182682 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.182973 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.183043 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.184046 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.184047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.184047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.184052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.185239 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.185646 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.185647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.185647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.185691 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.185887 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.185888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.185888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.185984 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.186034 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.186235 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.186294 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260929 06:28:49.186448 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.180619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.13.15.131:37371)
0929 06:28:49.180736 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.180745 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.180794 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.180981 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.180982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.181165 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:49.181236 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.181237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.182110 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.182111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.182605 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.182607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.183045 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.183045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.183093 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.183537 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:49.183544 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:49.183963 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:49.185343 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.185344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.185345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.185569 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.185750 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.185751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.185752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.185819 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.186047 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:49.186048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:49.186108 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.186217 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.186316 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:49.203748 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.197557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.13.15.190:35971)
0929 06:28:49.197610 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.197612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.197624 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:28:49.197719 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.197720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.197729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.197800 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.198355 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.198787 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.199464 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.199638 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.199717 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.201805 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.201807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.201807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.201812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.202936 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.203226 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.203227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.203227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.203273 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.203415 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.203415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.203415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.203495 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.203561 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.203641 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.203692 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:28:49.205199 14487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:28:49.206813 14487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:49.213040 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.209043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:44477 (local address 127.0.0.1:45846)
0929 06:28:49.209418 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.209427 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.209507 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.209768 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.209769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.209772 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.209817 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.209818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.211005 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.211006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.212078 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.212079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.212730 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.212806 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.212819 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.212901 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.212953 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":31,"thread_start_us":52,"threads_started":1}
I20260929 06:28:49.213150 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.208864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44111 (local address 127.0.0.1:37196)
0929 06:28:49.209007 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.209044 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.209114 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.209400 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.209403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.209405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.209491 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.209493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.210823 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.210824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.212047 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.212050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.212900 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.212953 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.212956 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.213013 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.213046 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260929 06:28:49.213377 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.209218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.13.15.130:44477)
0929 06:28:49.209361 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.209366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.209483 (+ 117us) server_negotiation.cc:407] Connection header received
0929 06:28:49.209600 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.209603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.209618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.209696 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.209899 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.210940 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212158 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.212626 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212723 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.212782 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.213229 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.213278 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260929 06:28:49.213510 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.208961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.13.15.129:44111)
0929 06:28:49.209141 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.209146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.209168 (+ 22us) server_negotiation.cc:407] Connection header received
0929 06:28:49.209204 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.209206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.209220 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.209324 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.209574 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.210732 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212130 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.212822 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.212891 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.212937 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.213027 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.213069 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260929 06:28:49.214018 14764 tablet_service.cc:1467] Processing CreateTablet for tablet 45126be6a6674235b924d0fa7ecadb9b (DEFAULT_TABLE table=test-table [id=7cae2095693b48e3a6cbc11cae87d031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.214304 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.208899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:34149 (local address 127.0.0.1:58600)
0929 06:28:49.209375 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.209385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.209452 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.209790 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.209790 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.209792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.209821 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.209821 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.211908 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.211909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.212984 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.212987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.214032 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.214033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.214095 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.214098 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.214183 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.214234 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260929 06:28:49.214274 14624 tablet_service.cc:1467] Processing CreateTablet for tablet 45126be6a6674235b924d0fa7ecadb9b (DEFAULT_TABLE table=test-table [id=7cae2095693b48e3a6cbc11cae87d031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.214329 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.209147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.13.15.131:34149)
0929 06:28:49.209344 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.209348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.209458 (+ 110us) server_negotiation.cc:407] Connection header received
0929 06:28:49.209603 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.209605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.209618 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.209696 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.211014 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.211831 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.213071 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.213957 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.214017 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.214066 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.214196 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.214236 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260929 06:28:49.214506 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45126be6a6674235b924d0fa7ecadb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.214614 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45126be6a6674235b924d0fa7ecadb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.215054 14901 tablet_service.cc:1467] Processing CreateTablet for tablet 45126be6a6674235b924d0fa7ecadb9b (DEFAULT_TABLE table=test-table [id=7cae2095693b48e3a6cbc11cae87d031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.215333 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45126be6a6674235b924d0fa7ecadb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.217360 14990 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Bootstrap starting.
I20260929 06:28:49.217850 14990 tablet_bootstrap.cc:654] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.218065 14990 log.cc:824] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:49.218245 14991 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Bootstrap starting.
I20260929 06:28:49.218446 14993 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Bootstrap starting.
I20260929 06:28:49.218890 14991 tablet_bootstrap.cc:654] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.219192 14990 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: No bootstrap required, opened a new log
I20260929 06:28:49.219244 14991 log.cc:824] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:49.219281 14990 ts_tablet_manager.cc:1397] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:49.219377 14993 tablet_bootstrap.cc:654] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.219643 14993 log.cc:824] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:49.220032 14991 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: No bootstrap required, opened a new log
I20260929 06:28:49.220103 14991 ts_tablet_manager.cc:1397] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 06:28:49.220611 14993 tablet_bootstrap.cc:492] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: No bootstrap required, opened a new log
I20260929 06:28:49.220672 14993 ts_tablet_manager.cc:1397] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:28:49.221364 14990 raft_consensus.cc:348] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.221527 14990 raft_consensus.cc:374] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.221563 14990 raft_consensus.cc:729] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bfa14a41a647e18553b16f6e4e49c3, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.221695 14990 consensus_queue.cc:260] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.221975 14830 heartbeater.cc:503] Master 127.13.15.190:35971 was elected leader, sending a full tablet report...
I20260929 06:28:49.222070 14991 raft_consensus.cc:348] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.222240 14991 raft_consensus.cc:374] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.222283 14991 raft_consensus.cc:729] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a816833c1e75457798af1dd72afd84a6, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.222394 14990 ts_tablet_manager.cc:1428] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:28:49.222420 14991 consensus_queue.cc:260] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.222664 14991 ts_tablet_manager.cc:1428] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 06:28:49.222808 14690 heartbeater.cc:503] Master 127.13.15.190:35971 was elected leader, sending a full tablet report...
I20260929 06:28:49.222849 14993 raft_consensus.cc:348] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.223016 14993 raft_consensus.cc:374] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.223055 14993 raft_consensus.cc:729] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bfe49d12eaa44b1b9ec4c9e283691e7, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.223304 14993 consensus_queue.cc:260] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.223623 14970 heartbeater.cc:503] Master 127.13.15.190:35971 was elected leader, sending a full tablet report...
I20260929 06:28:49.223671 14993 ts_tablet_manager.cc:1428] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 06:28:49.277895 14831 tablet.cc:2369] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:28:49.331877 14997 raft_consensus.cc:482] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:49.332026 14997 raft_consensus.cc:504] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.332440 14997 leader_election.cc:288] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.336220 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.332752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:42373 (local address 127.13.15.131:34149)
0929 06:28:49.332808 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.332810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.332818 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:49.332840 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.332841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.332868 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.332934 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.333142 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.333877 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335347 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.335845 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335915 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.335922 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.336090 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.336142 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:28:49.336210 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.332672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:36107 (local address 127.13.15.130:44477)
0929 06:28:49.332727 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.332730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.332879 (+ 149us) server_negotiation.cc:407] Connection header received
0929 06:28:49.332904 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.332905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.332914 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.332969 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.333141 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.333887 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335347 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.335849 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335915 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.335922 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.336090 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.336142 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 06:28:49.336242 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.332616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:34149 (local address 127.13.15.129:42373)
0929 06:28:49.332671 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.332683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.332725 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.332969 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.332970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.332972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.333057 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.333059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.333962 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.333962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.335265 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.335267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.335925 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.336008 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.336012 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.336064 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.336129 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":121}
I20260929 06:28:49.336262 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.332617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:44477 (local address 127.13.15.129:36107)
0929 06:28:49.332798 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.332807 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.332858 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.332983 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.332983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.332984 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.333066 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.333067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.333962 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.333962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.335267 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.335269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.335926 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.335926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.336008 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.336012 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.336064 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.336126 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":105,"thread_start_us":109,"threads_started":1}
I20260929 06:28:49.336649 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45126be6a6674235b924d0fa7ecadb9b" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" is_pre_election: true
I20260929 06:28:49.336649 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45126be6a6674235b924d0fa7ecadb9b" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" is_pre_election: true
I20260929 06:28:49.336793 14781 raft_consensus.cc:2454] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 0.
I20260929 06:28:49.336793 14924 raft_consensus.cc:2454] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 0.
I20260929 06:28:49.337029 14576 leader_election.cc:302] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0bfa14a41a647e18553b16f6e4e49c3, a816833c1e75457798af1dd72afd84a6; no voters:
I20260929 06:28:49.337176 14997 raft_consensus.cc:2781] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:49.337260 14997 raft_consensus.cc:482] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:49.337293 14997 raft_consensus.cc:3037] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.337966 14997 raft_consensus.cc:504] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.338097 14997 leader_election.cc:288] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 election: Requested vote from peers a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.338250 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45126be6a6674235b924d0fa7ecadb9b" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfa14a41a647e18553b16f6e4e49c3"
I20260929 06:28:49.338272 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45126be6a6674235b924d0fa7ecadb9b" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
I20260929 06:28:49.338322 14781 raft_consensus.cc:3037] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.338328 14924 raft_consensus.cc:3037] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.339215 14781 raft_consensus.cc:2454] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 1.
I20260929 06:28:49.339223 14924 raft_consensus.cc:2454] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 1.
I20260929 06:28:49.339388 14577 leader_election.cc:302] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bfe49d12eaa44b1b9ec4c9e283691e7, a816833c1e75457798af1dd72afd84a6; no voters:
I20260929 06:28:49.339494 14997 raft_consensus.cc:2781] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:49.339618 14997 raft_consensus.cc:686] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 1 LEADER]: Becoming Leader. State: Replica: a816833c1e75457798af1dd72afd84a6, State: Running, Role: LEADER
I20260929 06:28:49.339717 14997 consensus_queue.cc:237] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.340471 14486 catalog_manager.cc:5495] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a816833c1e75457798af1dd72afd84a6 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:49.356591 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.354011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.13.15.129:44111)
0929 06:28:49.354054 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.354056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.354107 (+ 51us) server_negotiation.cc:407] Connection header received
0929 06:28:49.354227 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.354228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.354247 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.354319 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:49.354467 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.354930 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.355725 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.355919 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.356011 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.356219 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.356312 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.356418 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.356469 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.356491 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:28:49.358870 14781 raft_consensus.cc:1261] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Refusing update from remote peer a816833c1e75457798af1dd72afd84a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.358870 14924 raft_consensus.cc:1261] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Refusing update from remote peer a816833c1e75457798af1dd72afd84a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.359200 14997 consensus_queue.cc:1035] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.359346 15000 consensus_queue.cc:1035] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.361992 15005 mvcc.cc:205] Tried to move back new op lower bound from 7334556997051273216 to 7334556996975947776. Current Snapshot: MvccSnapshot[applied={T|T < 7334556997051273216}]
I20260929 06:28:49.362370 15004 mvcc.cc:205] Tried to move back new op lower bound from 7334556997051273216 to 7334556996975947776. Current Snapshot: MvccSnapshot[applied={T|T < 7334556997051273216}]
I20260929 06:28:49.364718 14486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:49.364794 14486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:49.365525 14486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Sending DeleteTablet for 3 replicas of tablet 45126be6a6674235b924d0fa7ecadb9b
I20260929 06:28:49.365839 14624 tablet_service.cc:1514] Processing DeleteTablet for tablet 45126be6a6674235b924d0fa7ecadb9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37196
I20260929 06:28:49.365852 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet 45126be6a6674235b924d0fa7ecadb9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58600
I20260929 06:28:49.366039 14764 tablet_service.cc:1514] Processing DeleteTablet for tablet 45126be6a6674235b924d0fa7ecadb9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:45846
I20260929 06:28:49.366263 15013 tablet_replica.cc:330] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: stopping tablet replica
I20260929 06:28:49.366361 15013 raft_consensus.cc:2229] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:28:49.366320 14486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 06:28:49.366451 15015 tablet_replica.cc:330] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: stopping tablet replica
W20260929 06:28:49.366464 14486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:49.366472 15013 raft_consensus.cc:2258] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.366537 15015 raft_consensus.cc:2229] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:49.366600 15014 tablet_replica.cc:330] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: stopping tablet replica
I20260929 06:28:49.366631 15015 raft_consensus.cc:2258] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.366678 15014 raft_consensus.cc:2229] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:49.366995 15014 raft_consensus.cc:2258] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.367094 15015 ts_tablet_manager.cc:1905] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.367435 15013 ts_tablet_manager.cc:1905] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.367832 14901 tablet_service.cc:1467] Processing CreateTablet for tablet e83885e9b6b543369ac3925c63ddf378 (DEFAULT_TABLE table=test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.367936 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83885e9b6b543369ac3925c63ddf378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.368844 15015 ts_tablet_manager.cc:1918] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:28:49.368918 15015 log.cc:1196] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/45126be6a6674235b924d0fa7ecadb9b
I20260929 06:28:49.368961 15013 ts_tablet_manager.cc:1918] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:28:49.369022 15013 log.cc:1196] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/45126be6a6674235b924d0fa7ecadb9b
I20260929 06:28:49.369163 15015 ts_tablet_manager.cc:1939] T 45126be6a6674235b924d0fa7ecadb9b P a0bfa14a41a647e18553b16f6e4e49c3: Deleting consensus metadata
I20260929 06:28:49.369261 15013 ts_tablet_manager.cc:1939] T 45126be6a6674235b924d0fa7ecadb9b P a816833c1e75457798af1dd72afd84a6: Deleting consensus metadata
I20260929 06:28:49.369447 14993 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Bootstrap starting.
I20260929 06:28:49.369818 15014 ts_tablet_manager.cc:1905] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.369843 14993 tablet_bootstrap.cc:654] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.370463 14472 catalog_manager.cc:4841] TS a816833c1e75457798af1dd72afd84a6 (127.13.15.129:44111): tablet 45126be6a6674235b924d0fa7ecadb9b (table test-table [id=7cae2095693b48e3a6cbc11cae87d031]) successfully deleted
I20260929 06:28:49.370539 14472 catalog_manager.cc:4841] TS a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477): tablet 45126be6a6674235b924d0fa7ecadb9b (table test-table [id=7cae2095693b48e3a6cbc11cae87d031]) successfully deleted
I20260929 06:28:49.370705 14764 tablet_service.cc:1467] Processing CreateTablet for tablet e83885e9b6b543369ac3925c63ddf378 (DEFAULT_TABLE table=test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.370769 14624 tablet_service.cc:1467] Processing CreateTablet for tablet e83885e9b6b543369ac3925c63ddf378 (DEFAULT_TABLE table=test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:49.370808 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83885e9b6b543369ac3925c63ddf378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.370873 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83885e9b6b543369ac3925c63ddf378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.371141 14993 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: No bootstrap required, opened a new log
I20260929 06:28:49.371184 14993 ts_tablet_manager.cc:1397] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:49.371333 14993 raft_consensus.cc:348] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.371402 14993 raft_consensus.cc:374] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.371423 14993 raft_consensus.cc:729] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bfe49d12eaa44b1b9ec4c9e283691e7, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.371469 14993 consensus_queue.cc:260] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.371567 14993 ts_tablet_manager.cc:1428] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.372008 14990 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Bootstrap starting.
I20260929 06:28:49.372136 14991 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Bootstrap starting.
I20260929 06:28:49.372361 14990 tablet_bootstrap.cc:654] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.372509 14991 tablet_bootstrap.cc:654] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.372884 14990 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: No bootstrap required, opened a new log
I20260929 06:28:49.372929 14990 ts_tablet_manager.cc:1397] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:49.373075 14990 raft_consensus.cc:348] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.373147 14990 raft_consensus.cc:374] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.373178 14990 raft_consensus.cc:729] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bfa14a41a647e18553b16f6e4e49c3, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.373181 15014 ts_tablet_manager.cc:1918] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:28:49.373234 14990 consensus_queue.cc:260] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.373265 15014 log.cc:1196] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/45126be6a6674235b924d0fa7ecadb9b
I20260929 06:28:49.373334 14990 ts_tablet_manager.cc:1428] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:49.373694 14991 tablet_bootstrap.cc:492] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: No bootstrap required, opened a new log
I20260929 06:28:49.373759 14991 ts_tablet_manager.cc:1397] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260929 06:28:49.373881 14991 raft_consensus.cc:348] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.373960 14991 raft_consensus.cc:374] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.373981 14991 raft_consensus.cc:729] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a816833c1e75457798af1dd72afd84a6, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.374018 14991 consensus_queue.cc:260] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.374097 14991 ts_tablet_manager.cc:1428] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.374465 15014 ts_tablet_manager.cc:1939] T 45126be6a6674235b924d0fa7ecadb9b P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting consensus metadata
I20260929 06:28:49.374872 14473 catalog_manager.cc:4841] TS 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149): tablet 45126be6a6674235b924d0fa7ecadb9b (table test-table [id=7cae2095693b48e3a6cbc11cae87d031]) successfully deleted
W20260929 06:28:49.376881 14691 tablet.cc:2369] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:28:49.413976 14996 raft_consensus.cc:482] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:49.414055 14996 raft_consensus.cc:504] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.414428 14996 leader_election.cc:288] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a816833c1e75457798af1dd72afd84a6 (127.13.15.129:44111), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.417610 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.414585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:34149 (local address 127.13.15.130:52383)
0929 06:28:49.414634 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.414645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.414681 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.414920 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.414920 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.414922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.415007 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.415008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.415848 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.415849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.416816 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.416819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.417331 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.417331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.417411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.417414 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.417479 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.417518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 06:28:49.417621 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.414760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:52383 (local address 127.13.15.131:34149)
0929 06:28:49.414799 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.414801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.414809 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:49.414835 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.414836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.414844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.414891 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.415083 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.415768 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.416928 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.417241 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.417332 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.417340 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.417506 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.417549 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:28:49.417717 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.414642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:41239 (local address 127.13.15.129:44111)
0929 06:28:49.414683 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.414685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.414813 (+ 128us) server_negotiation.cc:407] Connection header received
0929 06:28:49.414913 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.414914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.414923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.414978 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:49.415115 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.415559 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.416902 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.417384 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.417453 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.417461 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.417606 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.417647 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:28:49.417958 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.414593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44111 (local address 127.13.15.130:41239)
0929 06:28:49.414735 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.414747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.414838 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.414991 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.414991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.414993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:49.415061 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.415062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.415639 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.415640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.416822 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.416823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.417475 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.417476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.417528 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.417531 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.417828 (+ 297us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.417866 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260929 06:28:49.417913 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83885e9b6b543369ac3925c63ddf378" candidate_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" is_pre_election: true
I20260929 06:28:49.417992 14924 raft_consensus.cc:2454] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0bfa14a41a647e18553b16f6e4e49c3 in term 0.
I20260929 06:28:49.418182 14717 leader_election.cc:302] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bfe49d12eaa44b1b9ec4c9e283691e7, a0bfa14a41a647e18553b16f6e4e49c3; no voters:
I20260929 06:28:49.418190 14644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83885e9b6b543369ac3925c63ddf378" candidate_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a816833c1e75457798af1dd72afd84a6" is_pre_election: true
I20260929 06:28:49.418291 14644 raft_consensus.cc:2454] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0bfa14a41a647e18553b16f6e4e49c3 in term 0.
I20260929 06:28:49.418318 14996 raft_consensus.cc:2781] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:49.418346 14996 raft_consensus.cc:482] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:49.418376 14996 raft_consensus.cc:3037] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.419090 14996 raft_consensus.cc:504] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.419212 14996 leader_election.cc:288] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [CANDIDATE]: Term 1 election: Requested vote from peers a816833c1e75457798af1dd72afd84a6 (127.13.15.129:44111), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.419327 14644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83885e9b6b543369ac3925c63ddf378" candidate_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a816833c1e75457798af1dd72afd84a6"
I20260929 06:28:49.419397 14644 raft_consensus.cc:3037] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.419425 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83885e9b6b543369ac3925c63ddf378" candidate_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
I20260929 06:28:49.419507 14924 raft_consensus.cc:3037] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.420037 14644 raft_consensus.cc:2454] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0bfa14a41a647e18553b16f6e4e49c3 in term 1.
I20260929 06:28:49.420189 14716 leader_election.cc:302] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0bfa14a41a647e18553b16f6e4e49c3, a816833c1e75457798af1dd72afd84a6; no voters:
I20260929 06:28:49.420224 14924 raft_consensus.cc:2454] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0bfa14a41a647e18553b16f6e4e49c3 in term 1.
I20260929 06:28:49.420301 14996 raft_consensus.cc:2781] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:49.420440 14996 raft_consensus.cc:686] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 LEADER]: Becoming Leader. State: Replica: a0bfa14a41a647e18553b16f6e4e49c3, State: Running, Role: LEADER
I20260929 06:28:49.420526 14996 consensus_queue.cc:237] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.421191 14486 catalog_manager.cc:5495] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130). New cstate: current_term: 1 leader_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } health_report { overall_health: UNKNOWN } } }
W20260929 06:28:49.427103 14971 tablet.cc:2369] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:49.454312 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.446833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35971 (local address 127.0.0.1:48432)
0929 06:28:49.447008 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.447020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.447095 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.447235 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.447236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.447618 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:49.447683 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.447685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.448300 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.448301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.449075 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.449078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.449425 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.449426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.449506 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.450018 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:49.450023 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:49.451477 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:49.453180 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.453185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.453187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.453374 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.453592 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:49.453593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:49.453593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:49.453675 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:49.453974 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:49.453980 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:49.454026 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.454081 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.454232 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260929 06:28:49.454429 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.446936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.13.15.190:35971)
0929 06:28:49.446984 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.446986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.447092 (+ 106us) server_negotiation.cc:407] Connection header received
0929 06:28:49.447155 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.447155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.447163 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.447213 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.447773 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.448224 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.449153 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.449350 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.449404 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.451599 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.451600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.451601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.451607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.453098 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.453466 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.453467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.453468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.453529 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.453742 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.453743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.453743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.453875 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.453943 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.454126 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.454312 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":24}
I20260929 06:28:49.456322 14486 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:48432:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 06:28:49.456539 14486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:49.458042 14624 tablet_service.cc:1467] Processing CreateTablet for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48da4b67d7742d89a3a228f1bd55599]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:49.458065 14901 tablet_service.cc:1467] Processing CreateTablet for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48da4b67d7742d89a3a228f1bd55599]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:49.458215 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.458215 14624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.459674 14991 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: Bootstrap starting.
I20260929 06:28:49.459728 14764 tablet_service.cc:1467] Processing CreateTablet for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48da4b67d7742d89a3a228f1bd55599]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:49.459861 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78f26ee7dfe4ad7b1fb569e2a474cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:49.460114 14991 tablet_bootstrap.cc:654] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.460933 14993 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Bootstrap starting.
I20260929 06:28:49.461023 14991 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: No bootstrap required, opened a new log
I20260929 06:28:49.461062 14991 ts_tablet_manager.cc:1397] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:49.461201 14991 raft_consensus.cc:348] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.461270 14991 raft_consensus.cc:374] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.461290 14991 raft_consensus.cc:729] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a816833c1e75457798af1dd72afd84a6, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.461293 14993 tablet_bootstrap.cc:654] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.461337 14991 consensus_queue.cc:260] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.461418 14991 ts_tablet_manager.cc:1428] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.461450 15000 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.461548 15000 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.461617 14990 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: Bootstrap starting.
I20260929 06:28:49.462042 14990 tablet_bootstrap.cc:654] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:49.462757 14990 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: No bootstrap required, opened a new log
I20260929 06:28:49.462795 14990 ts_tablet_manager.cc:1397] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:28:49.463044 14990 raft_consensus.cc:348] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.463099 14990 raft_consensus.cc:374] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.463114 14990 raft_consensus.cc:729] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bfa14a41a647e18553b16f6e4e49c3, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.463165 14990 consensus_queue.cc:260] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.463291 14990 ts_tablet_manager.cc:1428] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:49.463411 14996 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.463479 14996 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.463945 14993 tablet_bootstrap.cc:492] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: No bootstrap required, opened a new log
I20260929 06:28:49.463991 14993 ts_tablet_manager.cc:1397] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 06:28:49.464124 14993 raft_consensus.cc:348] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.464184 14993 raft_consensus.cc:374] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:49.464197 14993 raft_consensus.cc:729] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bfe49d12eaa44b1b9ec4c9e283691e7, State: Initialized, Role: FOLLOWER
I20260929 06:28:49.464262 14993 consensus_queue.cc:260] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.464349 14993 ts_tablet_manager.cc:1428] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:49.464403 14998 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.464473 14998 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.705044 15000 raft_consensus.cc:482] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:49.705147 15000 raft_consensus.cc:504] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.705400 15000 leader_election.cc:288] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.705626 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78f26ee7dfe4ad7b1fb569e2a474cc6" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" is_pre_election: true
I20260929 06:28:49.705641 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78f26ee7dfe4ad7b1fb569e2a474cc6" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" is_pre_election: true
I20260929 06:28:49.705725 14781 raft_consensus.cc:2454] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 0.
I20260929 06:28:49.705727 14924 raft_consensus.cc:2454] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 0.
I20260929 06:28:49.705909 14576 leader_election.cc:302] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0bfa14a41a647e18553b16f6e4e49c3, a816833c1e75457798af1dd72afd84a6; no voters:
I20260929 06:28:49.705997 15000 raft_consensus.cc:2781] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:49.706037 15000 raft_consensus.cc:482] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:49.706068 15000 raft_consensus.cc:3037] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.706822 15000 raft_consensus.cc:504] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.706959 15000 leader_election.cc:288] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 election: Requested vote from peers a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477), 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149)
I20260929 06:28:49.707119 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78f26ee7dfe4ad7b1fb569e2a474cc6" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfa14a41a647e18553b16f6e4e49c3"
I20260929 06:28:49.707124 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78f26ee7dfe4ad7b1fb569e2a474cc6" candidate_uuid: "a816833c1e75457798af1dd72afd84a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7"
I20260929 06:28:49.707190 14781 raft_consensus.cc:3037] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.707212 14924 raft_consensus.cc:3037] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:49.707783 14924 raft_consensus.cc:2454] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 1.
I20260929 06:28:49.707965 14577 leader_election.cc:302] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bfe49d12eaa44b1b9ec4c9e283691e7, a816833c1e75457798af1dd72afd84a6; no voters:
I20260929 06:28:49.707988 14781 raft_consensus.cc:2454] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a816833c1e75457798af1dd72afd84a6 in term 1.
I20260929 06:28:49.708081 15000 raft_consensus.cc:2781] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:49.708130 15000 raft_consensus.cc:686] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [term 1 LEADER]: Becoming Leader. State: Replica: a816833c1e75457798af1dd72afd84a6, State: Running, Role: LEADER
I20260929 06:28:49.708182 15000 consensus_queue.cc:237] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } }
I20260929 06:28:49.708444 15009 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: TxnStatusTablet state changed. Reason: New leader a816833c1e75457798af1dd72afd84a6. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.708537 15009 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:49.708676 15032 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 06:28:49.708815 15032 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 06:28:49.708825 14486 catalog_manager.cc:5495] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a816833c1e75457798af1dd72afd84a6 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:49.749029 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.746816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.13.15.129:44111)
0929 06:28:49.746860 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.746862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.746871 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:28:49.746938 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.746939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.746950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.747020 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:49.747124 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.747573 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.748354 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.748534 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.748604 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.748776 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.748815 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.748873 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.748938 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.748954 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:28:49.749042 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.746736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44111 (local address 127.0.0.1:37212)
0929 06:28:49.746804 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.746815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.746864 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.747027 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.747028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.747029 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:49.747075 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.747076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.747650 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.747651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.748263 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.748265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.748601 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.748602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.748661 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.748681 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.748874 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.748882 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.748933 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.748978 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 06:28:49.749930 14924 raft_consensus.cc:1261] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Refusing update from remote peer a816833c1e75457798af1dd72afd84a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.749938 14781 raft_consensus.cc:1261] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Refusing update from remote peer a816833c1e75457798af1dd72afd84a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.750188 15009 consensus_queue.cc:1035] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.750316 15000 consensus_queue.cc:1035] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.750756 14996 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: TxnStatusTablet state changed. Reason: New leader a816833c1e75457798af1dd72afd84a6. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.750849 14996 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.750816 14998 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: TxnStatusTablet state changed. Reason: New leader a816833c1e75457798af1dd72afd84a6. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.750893 14998 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.751510 15000 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.751602 15000 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:49.751881 15009 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.751900 14996 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.751932 15003 mvcc.cc:205] Tried to move back new op lower bound from 7334556998654062592 to 7334556998484992000. Current Snapshot: MvccSnapshot[applied={T|T < 7334556998654062592 or (T in {7334556998654062592})}]
I20260929 06:28:49.751964 15009 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a816833c1e75457798af1dd72afd84a6: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:49.751976 14996 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P a0bfa14a41a647e18553b16f6e4e49c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.752534 14998 tablet_replica.cc:439] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a816833c1e75457798af1dd72afd84a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 } } peers { permanent_uuid: "a0bfa14a41a647e18553b16f6e4e49c3" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 44477 } } peers { permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 } } }
I20260929 06:28:49.752619 14998 tablet_replica.cc:442] T a78f26ee7dfe4ad7b1fb569e2a474cc6 P 9bfe49d12eaa44b1b9ec4c9e283691e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:49.756408 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.753881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.13.15.130:44477)
0929 06:28:49.753916 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.753917 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.753925 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:49.754008 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.754009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.754023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.754077 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:49.754237 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.754734 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.755761 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.755931 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.755974 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.756137 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.756228 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.756273 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.756342 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.756357 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:28:49.761111 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.758708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:38189 (local address 127.13.15.129:44111)
0929 06:28:49.758774 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.758776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.758923 (+ 147us) server_negotiation.cc:407] Connection header received
0929 06:28:49.758973 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.758974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.758980 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.759019 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:49.759135 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.759577 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.760559 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.760717 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.760764 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.760942 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.760962 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.760995 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.761042 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.761053 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:28:49.761173 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.758615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44111 (local address 127.13.15.130:38189)
0929 06:28:49.758850 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.758861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.758920 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.759031 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.759032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.759033 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:49.759084 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.759085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.759644 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.759644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.760483 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.760486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.760802 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.760803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.760860 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.760872 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.760997 (+ 125us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.760998 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.761041 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.761079 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":119,"threads_started":1}
I20260929 06:28:49.763185 14924 raft_consensus.cc:1261] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Refusing update from remote peer a0bfa14a41a647e18553b16f6e4e49c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.763289 14644 raft_consensus.cc:1261] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Refusing update from remote peer a0bfa14a41a647e18553b16f6e4e49c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:49.763582 15020 consensus_queue.cc:1035] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a816833c1e75457798af1dd72afd84a6" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.763695 14996 consensus_queue.cc:1035] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bfe49d12eaa44b1b9ec4c9e283691e7" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:49.771781 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.769200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:44477 (local address 127.13.15.129:52357)
0929 06:28:49.769333 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:49.769342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:49.769399 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:49.769510 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:49.769510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:49.769511 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:49.769537 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.769537 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.770318 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.770319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.771091 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:49.771093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.771492 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.771492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:49.771527 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.771537 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.771672 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.771673 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:28:49.771702 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 06:28:49.771731 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260929 06:28:49.771795 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.769227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:52357 (local address 127.13.15.130:44477)
0929 06:28:49.769260 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.769262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.769403 (+ 141us) server_negotiation.cc:407] Connection header received
0929 06:28:49.769429 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.769430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.769438 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.769508 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:49.769574 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.770246 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.771152 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.771418 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.771489 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.771590 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:49.771618 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:49.771673 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.771706 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.771723 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:28:49.777894 14486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:49.778009 14486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:49.778609 14486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31e1eec403aa44299ede41e63e26bf39: Sending DeleteTablet for 3 replicas of tablet e83885e9b6b543369ac3925c63ddf378
I20260929 06:28:49.778824 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet e83885e9b6b543369ac3925c63ddf378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58600
I20260929 06:28:49.778928 14764 tablet_service.cc:1514] Processing DeleteTablet for tablet e83885e9b6b543369ac3925c63ddf378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:45846
I20260929 06:28:49.778927 14624 tablet_service.cc:1514] Processing DeleteTablet for tablet e83885e9b6b543369ac3925c63ddf378 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:49 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37196
I20260929 06:28:49.779001 15015 tablet_replica.cc:330] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: stopping tablet replica
I20260929 06:28:49.779019 15014 tablet_replica.cc:330] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: stopping tablet replica
I20260929 06:28:49.779050 15015 raft_consensus.cc:2229] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:28:49.779071 15014 raft_consensus.cc:2229] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:49.779114 15014 raft_consensus.cc:2258] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.779121 15015 raft_consensus.cc:2258] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.779177 15013 tablet_replica.cc:330] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: stopping tablet replica
I20260929 06:28:49.779237 15013 raft_consensus.cc:2229] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:49.779274 15013 raft_consensus.cc:2258] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:49.779615 15013 ts_tablet_manager.cc:1905] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.779922 15015 ts_tablet_manager.cc:1905] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.781404 15015 ts_tablet_manager.cc:1918] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:49.781464 15015 log.cc:1196] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/e83885e9b6b543369ac3925c63ddf378
I20260929 06:28:49.781490 15014 ts_tablet_manager.cc:1905] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:49.781692 15015 ts_tablet_manager.cc:1939] T e83885e9b6b543369ac3925c63ddf378 P a0bfa14a41a647e18553b16f6e4e49c3: Deleting consensus metadata
I20260929 06:28:49.781874 15013 ts_tablet_manager.cc:1918] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:49.781922 15013 log.cc:1196] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/e83885e9b6b543369ac3925c63ddf378
I20260929 06:28:49.782074 14472 catalog_manager.cc:4841] TS a0bfa14a41a647e18553b16f6e4e49c3 (127.13.15.130:44477): tablet e83885e9b6b543369ac3925c63ddf378 (table test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]) successfully deleted
I20260929 06:28:49.782115 15013 ts_tablet_manager.cc:1939] T e83885e9b6b543369ac3925c63ddf378 P a816833c1e75457798af1dd72afd84a6: Deleting consensus metadata
I20260929 06:28:49.782429 14472 catalog_manager.cc:4841] TS a816833c1e75457798af1dd72afd84a6 (127.13.15.129:44111): tablet e83885e9b6b543369ac3925c63ddf378 (table test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]) successfully deleted
I20260929 06:28:49.782897 15014 ts_tablet_manager.cc:1918] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:49.782968 15014 log.cc:1196] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/e83885e9b6b543369ac3925c63ddf378
I20260929 06:28:49.783195 15014 ts_tablet_manager.cc:1939] T e83885e9b6b543369ac3925c63ddf378 P 9bfe49d12eaa44b1b9ec4c9e283691e7: Deleting consensus metadata
I20260929 06:28:49.783499 14473 catalog_manager.cc:4841] TS 9bfe49d12eaa44b1b9ec4c9e283691e7 (127.13.15.131:34149): tablet e83885e9b6b543369ac3925c63ddf378 (table test-table [id=ea9ad2c384ae4173b84d4d08f7d13465]) successfully deleted
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[14442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:28:49.793941 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.779873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.13.15.129:44111)
0929 06:28:49.781286 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.781287 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.786314 (+ 5027us) server_negotiation.cc:407] Connection header received
0929 06:28:49.786386 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.786387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.786397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.786440 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.786975 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.787411 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.788214 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.788411 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.788490 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.791057 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.791064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.791065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.791071 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.792766 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.793113 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.793116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.793116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.793181 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.793412 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.793413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.793413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.793550 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.793696 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.793765 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.793841 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1395}
W20260929 06:28:49.794252 14684 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37216
I20260929 06:28:49.801380 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:49.795056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.13.15.190:35971)
0929 06:28:49.795095 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:49.795097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:49.795327 (+ 230us) server_negotiation.cc:407] Connection header received
0929 06:28:49.795398 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:49.795398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:49.795408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:49.795461 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:49.796015 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.796658 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.797421 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:49.797730 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:49.797802 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:49.798970 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:49.798971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:49.798972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:49.798978 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:49.800307 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.800640 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.800641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.800642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.800692 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:49.800885 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:49.800886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:49.800887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:49.800994 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:49.801092 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:49.801189 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 06:28:49.801263 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 06:28:49.801553 14486 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48446
I20260929 06:28:49.802135 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14559
I20260929 06:28:49.812136 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14699
I20260929 06:28:49.817402 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14839
I20260929 06:28:49.821595 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 14455
2026-09-29T06:28:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2384 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[15054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[15054](info): set up 2 sockets
Sep 29 06:28:49 dist-test-slave-sm0p krb5kdc[15054](info): commencing operation
krb5kdc: starting...
W20260929 06:28:50.829890 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:50 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:50.944381 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44753
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:51.020231 15067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:51.020434 15067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:51.020455 15067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:51.021462 15067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:51.021493 15067 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 06:28:51.021507 15067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:51.021518 15067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:51.021530 15067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:51.021543 15067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:51.022914 15067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44753
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:51.023092 15067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:51.023241 15067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:51.027969 15075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.027998 15073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.027998 15072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.028178 15067 server_base.cc:1034] running on GCE node
I20260929 06:28:51.028411 15067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:51.028705 15067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:51.029866 15067 hybrid_clock.cc:648] HybridClock initialized: now 1790663331029860 us; error 45 us; skew 500 ppm
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:51.032436 15067 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:51.032824 15067 webserver.cc:460] Webserver started at http://127.13.15.190:45279/ using document root <none> and password file <none>
I20260929 06:28:51.033033 15067 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:51.033097 15067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:51.033237 15067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:51.034303 15067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "e52a3e3d881a46628793ee1c809bcbf2"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "5d462969077fe7af22a4b83e9b2f157a"
server_key_iv: "545a7b924315d73d3a5ec75ccd7dbd8f"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.034649 15067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "e52a3e3d881a46628793ee1c809bcbf2"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "5d462969077fe7af22a4b83e9b2f157a"
server_key_iv: "545a7b924315d73d3a5ec75ccd7dbd8f"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.035879 15067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:28:51.036572 15082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.036736 15067 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.036836 15067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "e52a3e3d881a46628793ee1c809bcbf2"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "5d462969077fe7af22a4b83e9b2f157a"
server_key_iv: "545a7b924315d73d3a5ec75ccd7dbd8f"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.036943 15067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:51.052609 15067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:51.054936 15067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:51.055092 15067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:51.059525 15067 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44753
I20260929 06:28:51.059577 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44753 every 8 connection(s)
I20260929 06:28:51.059873 15067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:51.060447 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.062711 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Bootstrap starting.
I20260929 06:28:51.063279 15145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.063575 15145 log.cc:824] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:51.064217 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: No bootstrap required, opened a new log
I20260929 06:28:51.066365 15145 raft_consensus.cc:348] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } }
I20260929 06:28:51.066493 15145 raft_consensus.cc:374] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.066515 15145 raft_consensus.cc:729] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52a3e3d881a46628793ee1c809bcbf2, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.066682 15145 consensus_queue.cc:260] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } }
I20260929 06:28:51.066743 15145 raft_consensus.cc:388] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:51.066782 15145 raft_consensus.cc:482] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:51.066833 15145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.067338 15145 raft_consensus.cc:504] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } }
I20260929 06:28:51.067446 15145 leader_election.cc:302] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e52a3e3d881a46628793ee1c809bcbf2; no voters:
I20260929 06:28:51.067627 15145 leader_election.cc:288] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:51.067689 15148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:51.067847 15148 raft_consensus.cc:686] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [term 1 LEADER]: Becoming Leader. State: Replica: e52a3e3d881a46628793ee1c809bcbf2, State: Running, Role: LEADER
I20260929 06:28:51.067905 15145 sys_catalog.cc:564] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:51.067929 15148 consensus_queue.cc:237] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } }
I20260929 06:28:51.068262 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e52a3e3d881a46628793ee1c809bcbf2. Latest consensus state: current_term: 1 leader_uuid: "e52a3e3d881a46628793ee1c809bcbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } } }
I20260929 06:28:51.068266 15149 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e52a3e3d881a46628793ee1c809bcbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52a3e3d881a46628793ee1c809bcbf2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44753 } } }
I20260929 06:28:51.068367 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:51.068373 15149 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:51.068571 15154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:51.069123 15154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:28:51.069177 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15067
I20260929 06:28:51.069278 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:51.069468 13374 external_mini_cluster.cc:1428] Setting key 776c03432d55cd85088e9214b1053f50
I20260929 06:28:51.070884 15154 catalog_manager.cc:1347] Generated new cluster ID: e5db5dba388c42e7b07ae31fa68a5f55
I20260929 06:28:51.070938 15154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.079421 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.069931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.13.15.190:44753)
0929 06:28:51.070072 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.070080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.070112 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:28:51.070253 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.070260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.070323 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.070468 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.071159 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.072034 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.072806 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.073249 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.073374 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.075711 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.075715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.075715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.075727 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.077728 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.078305 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.078308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.078308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.078420 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.078614 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.078615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.078616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.078840 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.079006 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.079121 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.079261 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260929 06:28:51.079679 15154 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:51.080231 15154 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:51.082393 15154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Generated new TSK 0
I20260929 06:28:51.082556 15154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:28:51.108526 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44753
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:51.188174 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:51.188401 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:51.188442 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:51.189595 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:51.189657 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:51.189693 15171 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 06:28:51.189741 15171 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:51.189774 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:28:51.191447 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:44753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:51.191690 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:51.191919 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:51.197710 15177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.197739 15171 server_base.cc:1034] running on GCE node
W20260929 06:28:51.197710 15179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.197824 15176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.198172 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:51.198426 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:51.199591 15171 hybrid_clock.cc:648] HybridClock initialized: now 1790663331199566 us; error 42 us; skew 500 ppm
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:51.202183 15171 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:28:51.202643 15171 webserver.cc:460] Webserver started at http://127.13.15.129:43443/ using document root <none> and password file <none>
I20260929 06:28:51.202875 15171 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:51.202919 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:51.203065 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:51.204138 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "93fa3708bf6b4a89878b78d478d49bb1"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "1b313a1f7bd0d87586f271b7dcdfd849"
server_key_iv: "35f7fefd1373a6c64639fb71e3a22a54"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.204495 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "93fa3708bf6b4a89878b78d478d49bb1"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "1b313a1f7bd0d87586f271b7dcdfd849"
server_key_iv: "35f7fefd1373a6c64639fb71e3a22a54"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.205840 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:51.206700 15186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.206928 15171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:28:51.207062 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "93fa3708bf6b4a89878b78d478d49bb1"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "1b313a1f7bd0d87586f271b7dcdfd849"
server_key_iv: "35f7fefd1373a6c64639fb71e3a22a54"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.207159 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:51.233433 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:51.236052 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:51.236261 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:51.236887 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:51.236933 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.236959 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:51.237008 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.244913 15171 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:35769
I20260929 06:28:51.245136 15301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:35769 every 8 connection(s)
I20260929 06:28:51.245398 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:28:51.247430 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.237287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:35669 (local address 127.13.15.190:44753)
0929 06:28:51.237354 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.237356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.237598 (+ 242us) server_negotiation.cc:407] Connection header received
0929 06:28:51.238118 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.238120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.238163 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.238213 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.238933 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.239751 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.241097 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.241475 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.241575 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.244417 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.244418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.244419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.244430 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.246355 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.246693 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.246694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.246695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.246748 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.246943 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.246944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.246944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.247040 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.247091 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.247300 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.247368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 06:28:51.247684 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.237294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.129:35669)
0929 06:28:51.237479 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.237514 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.238008 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.238287 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.238298 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.238556 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.238805 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.238810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.239873 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.239874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.241023 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.241026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.241576 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.241577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.241734 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.242308 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.242316 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.244316 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.246426 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.246430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.246432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.246637 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.246810 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.246810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.246811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.246903 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.247092 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.247095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.247204 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.247278 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.247402 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260929 06:28:51.248284 15302 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44753
I20260929 06:28:51.248381 15302 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:51.248551 15302 heartbeater.cc:511] Master 127.13.15.190:44753 requested a full tablet report, sending...
I20260929 06:28:51.249055 15098 ts_manager.cc:194] Registered new tserver with Master: 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769)
I20260929 06:28:51.249925 15098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:35669
I20260929 06:28:51.253785 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15171
I20260929 06:28:51.253923 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:28:51.254132 13374 external_mini_cluster.cc:1428] Setting key 311b103551faf25facd85b9df6f5f263
I20260929 06:28:51.254499 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.249346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:43663 (local address 127.13.15.190:44753)
0929 06:28:51.249388 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.249390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.249469 (+ 79us) server_negotiation.cc:407] Connection header received
0929 06:28:51.249529 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.249530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.249539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.249591 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.249867 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.250438 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.251205 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.251376 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.251427 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.252218 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.252219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.252219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.252224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.253322 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.253603 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.253604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.253604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.253649 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.253954 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.253955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.253956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.254084 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.254177 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.254335 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.254414 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:28:51.254508 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.249276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.129:43663)
0929 06:28:51.249411 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.249421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.249476 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.249603 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.249603 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.249742 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.249802 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.249804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.250527 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.250528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.251135 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.251137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.251427 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.251427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.251473 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.251882 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.251887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.252154 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.253389 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.253390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.253390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.253541 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.253722 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.253723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.253723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.253792 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.254151 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.254152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.254202 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.254278 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.254405 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:28:51.278827 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44753
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:51.374459 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:51.374642 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:51.374666 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:51.375794 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:51.375833 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:51.375877 15311 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 06:28:51.375903 15311 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:51.375933 15311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:28:51.377328 15311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:44753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:51.377562 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:51.377787 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:51.382601 15319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.382663 15311 server_base.cc:1034] running on GCE node
W20260929 06:28:51.382632 15317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.382589 15316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.383016 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:51.383246 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:51.384428 15311 hybrid_clock.cc:648] HybridClock initialized: now 1790663331384424 us; error 36 us; skew 500 ppm
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:51.386854 15311 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:28:51.387265 15311 webserver.cc:460] Webserver started at http://127.13.15.130:33791/ using document root <none> and password file <none>
I20260929 06:28:51.387497 15311 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:51.387542 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:51.387697 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:51.388687 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "7c3ac9cbf8314d20863537e39646a80c"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "f29a4adf5b966b275f6131041bce7eb0"
server_key_iv: "383471f9f335909a24adcfa7046cb53c"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.389025 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "7c3ac9cbf8314d20863537e39646a80c"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "f29a4adf5b966b275f6131041bce7eb0"
server_key_iv: "383471f9f335909a24adcfa7046cb53c"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.390372 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 06:28:51.391095 15326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.391286 15311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:51.391377 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "7c3ac9cbf8314d20863537e39646a80c"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "f29a4adf5b966b275f6131041bce7eb0"
server_key_iv: "383471f9f335909a24adcfa7046cb53c"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.391446 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:51.416854 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:51.418642 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:51.418749 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:51.419165 15311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:51.419199 15311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.419260 15311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:51.419291 15311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.426522 15311 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:34647
I20260929 06:28:51.426641 15441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:34647 every 8 connection(s)
I20260929 06:28:51.426929 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:28:51.428718 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.419698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:56873 (local address 127.13.15.190:44753)
0929 06:28:51.419776 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.419778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.419937 (+ 159us) server_negotiation.cc:407] Connection header received
0929 06:28:51.420421 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.420423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.420437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.420527 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.421211 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.422044 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.423066 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.423302 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.423385 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.426031 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.426033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.426033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.426041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.427651 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.427998 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.427998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.427999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.428046 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.428215 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.428215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.428216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.428310 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.428357 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.428601 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.428658 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:28:51.428867 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.419654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.130:56873)
0929 06:28:51.419810 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.419848 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.420321 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.420617 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.420621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.420928 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.421105 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.421110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.422163 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.422164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.422966 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.422968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.423390 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.423391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.423525 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.424000 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.424008 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.425955 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.427719 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.427723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.427725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.427931 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.428086 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.428086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.428086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.428164 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.428364 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.428369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.428475 (+ 106us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.428561 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.428669 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260929 06:28:51.429323 15442 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44753
I20260929 06:28:51.429421 15442 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:51.429585 15442 heartbeater.cc:511] Master 127.13.15.190:44753 requested a full tablet report, sending...
I20260929 06:28:51.429970 15099 ts_manager.cc:194] Registered new tserver with Master: 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647)
I20260929 06:28:51.430456 15099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:56873
I20260929 06:28:51.433311 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15311
I20260929 06:28:51.433411 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:28:51.433604 13374 external_mini_cluster.cc:1428] Setting key d8b060f571bc410d754b1b2e31e4549a
I20260929 06:28:51.435856 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.430412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:41837 (local address 127.13.15.190:44753)
0929 06:28:51.430451 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.430453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.430524 (+ 71us) server_negotiation.cc:407] Connection header received
0929 06:28:51.430669 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.430670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.430680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.430749 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.431352 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.431778 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.432403 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.432573 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.432619 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.433475 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.433477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.433477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.433484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.434774 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.435103 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.435105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.435105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.435159 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.435351 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.435352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.435353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.435479 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.435554 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.435698 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.435764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:28:51.435856 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.430329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.130:41837)
0929 06:28:51.430457 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.430468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.430528 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.431097 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.431098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.431237 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.431287 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.431288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.431861 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.431861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.432340 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.432341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.432631 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.432631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.432681 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.433093 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.433098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.433371 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.434873 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.434873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.434874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.435034 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.435222 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.435223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.435223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.435276 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.435547 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.435548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.435590 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.435676 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.435766 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:28:51.456966 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44753
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:51.540890 15451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:51.541115 15451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:51.541157 15451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:51.542443 15451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:51.542507 15451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:51.542546 15451 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 06:28:51.542629 15451 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:28:51.542688 15451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:28:51.544170 15451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:44753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:51.544354 15451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:51.544526 15451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:28:51.549575 15451 server_base.cc:1034] running on GCE node
W20260929 06:28:51.549510 15459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.549512 15457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:51.549506 15456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:51.549916 15451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:51.550181 15451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:51.551333 15451 hybrid_clock.cc:648] HybridClock initialized: now 1790663331551322 us; error 27 us; skew 500 ppm
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:51.553745 15451 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:28:51.554105 15451 webserver.cc:460] Webserver started at http://127.13.15.131:46253/ using document root <none> and password file <none>
I20260929 06:28:51.554462 15451 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:51.554512 15451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:51.554656 15451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:51.555658 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "85d214e3f6374a30bd6721d5951a68bd"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "4fb196eba2a80297b68db9a59b3e9af1"
server_key_iv: "7df9641d0abc5b0da1731b1eb3a75044"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.556020 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "85d214e3f6374a30bd6721d5951a68bd"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "4fb196eba2a80297b68db9a59b3e9af1"
server_key_iv: "7df9641d0abc5b0da1731b1eb3a75044"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.557263 15451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:28:51.557891 15466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.558105 15451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.558248 15451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "85d214e3f6374a30bd6721d5951a68bd"
format_stamp: "Formatted at 2026-09-29 06:28:51 on dist-test-slave-sm0p"
server_key: "4fb196eba2a80297b68db9a59b3e9af1"
server_key_iv: "7df9641d0abc5b0da1731b1eb3a75044"
server_key_version: "encryptionkey@0"
I20260929 06:28:51.558331 15451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:51.597216 15451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:51.599812 15451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:51.599972 15451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:51.600472 15451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:51.600539 15451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.600584 15451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:51.600620 15451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.608280 15451 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:35149
I20260929 06:28:51.608560 15581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:35149 every 8 connection(s)
I20260929 06:28:51.608742 15451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:28:51.610903 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.600917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:53019 (local address 127.13.15.190:44753)
0929 06:28:51.600993 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.600996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.601186 (+ 190us) server_negotiation.cc:407] Connection header received
0929 06:28:51.601705 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.601706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.601717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.601770 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.602591 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.603316 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.605018 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.605339 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.605425 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.607970 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.607972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.607972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.607980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.609567 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.609957 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.609957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.609958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.610003 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.610282 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.610282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.610282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.610378 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.610425 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.610706 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.610776 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 06:28:51.610949 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.600901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.131:53019)
0929 06:28:51.601066 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.601102 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.601594 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.601871 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.601875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.602264 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.602492 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.602496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.603436 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.603437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.604902 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.604907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.605472 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.605473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.605644 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.606201 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.606210 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.607874 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.609635 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.609639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.609641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.609884 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.610041 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.610041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.610041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.610219 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.610419 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.610423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.610531 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.610662 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.610797 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260929 06:28:51.611490 15582 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44753
I20260929 06:28:51.611583 15582 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:51.611809 15582 heartbeater.cc:511] Master 127.13.15.190:44753 requested a full tablet report, sending...
I20260929 06:28:51.612205 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15451
I20260929 06:28:51.612294 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:28:51.612491 13374 external_mini_cluster.cc:1428] Setting key 659bbcc1888228bd9ca7938fb114b0db
I20260929 06:28:51.612728 15098 ts_manager.cc:194] Registered new tserver with Master: 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:51.613243 15098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:53019
I20260929 06:28:51.613286 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:28:51.618924 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.612619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.13.15.131:49797)
0929 06:28:51.612749 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.612761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.612844 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.613036 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.613037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.613251 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.613312 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.613313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.614485 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.614486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.614972 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.614973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.615405 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.615405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.615487 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.616047 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.616051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.616463 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.617946 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.617947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.617948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.618195 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.618413 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.618414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.618414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.618463 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.618722 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.618723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.618754 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.618800 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.618866 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260929 06:28:51.619113 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.612616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:49797 (local address 127.13.15.190:44753)
0929 06:28:51.612665 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.612667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.612818 (+ 151us) server_negotiation.cc:407] Connection header received
0929 06:28:51.612922 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.612923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.612933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.612965 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.613578 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.614389 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.615085 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.615294 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.615404 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.616549 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.616550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.616550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.616556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.617865 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.618267 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.618268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.618269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.618331 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.618525 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.618526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.618526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.618660 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.618728 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.618825 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.618898 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.636269 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.629707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.13.15.190:44753)
0929 06:28:51.629742 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.629744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.629752 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:51.629834 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.629835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.629844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.629914 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.630500 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.630972 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.631727 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.631910 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.631978 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.634078 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.634080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.634080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.634088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.635206 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.635523 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.635524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.635525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.635578 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.635783 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.635784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.635784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.635884 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.635990 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.636104 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.636174 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:28:51.637567 15098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:28:51.639086 15098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:51.645308 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35769 (local address 127.0.0.1:58518)
0929 06:28:51.641320 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.641362 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.641424 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.641747 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.641751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.641754 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.641813 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.641815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.643201 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.643202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.644159 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.644161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.645028 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.645029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.645090 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645093 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.645129 (+ 36us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.645186 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":52,"threads_started":1}
I20260929 06:28:51.645318 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.13.15.130:34647)
0929 06:28:51.641732 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.641736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.641744 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:51.641775 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.641777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.641790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.642012 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.642192 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.643279 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644432 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.644909 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644995 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645048 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.645189 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.645225 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260929 06:28:51.645391 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.13.15.131:35149)
0929 06:28:51.641492 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.641497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.641508 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:28:51.641618 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.641621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.641637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.641691 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.641922 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.642831 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644155 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.644904 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644977 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645037 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.645227 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.645266 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
I20260929 06:28:51.645527 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35149 (local address 127.0.0.1:57828)
0929 06:28:51.641456 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.641467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.641516 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.641789 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.641790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.641794 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.641850 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.641851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.642910 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.642911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.644072 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.644074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.645063 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.645064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.645119 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645123 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.645166 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.645192 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":81,"threads_started":1}
I20260929 06:28:51.645795 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.13.15.129:35769)
0929 06:28:51.641595 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.641599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.641609 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:28:51.641643 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.641645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.641661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.641726 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.641897 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.643109 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644265 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.644943 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.645257 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645327 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.645644 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.645689 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":77,"threads_started":1}
I20260929 06:28:51.646389 15516 tablet_service.cc:1467] Processing CreateTablet for tablet 31fa09b5529a4b349274be3adf915aee (DEFAULT_TABLE table=test-table [id=37eb1dd994fa45dcb8805906168c8b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.646824 15516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fa09b5529a4b349274be3adf915aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.646996 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.641353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.0.0.1:47636)
0929 06:28:51.641549 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.641558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.641601 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.642065 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.642066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.642067 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.642098 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.642099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.643385 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.643386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.644354 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.644355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.644994 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.644995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.645066 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.645078 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.645174 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.645214 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":68,"threads_started":1}
I20260929 06:28:51.647295 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 31fa09b5529a4b349274be3adf915aee (DEFAULT_TABLE table=test-table [id=37eb1dd994fa45dcb8805906168c8b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.647543 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fa09b5529a4b349274be3adf915aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.649128 15376 tablet_service.cc:1467] Processing CreateTablet for tablet 31fa09b5529a4b349274be3adf915aee (DEFAULT_TABLE table=test-table [id=37eb1dd994fa45dcb8805906168c8b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.649471 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fa09b5529a4b349274be3adf915aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.649925 15605 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Bootstrap starting.
I20260929 06:28:51.650564 15606 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Bootstrap starting.
I20260929 06:28:51.650878 15605 tablet_bootstrap.cc:654] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.651199 15605 log.cc:824] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:51.651225 15606 tablet_bootstrap.cc:654] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.651551 15606 log.cc:824] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:51.651981 15605 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: No bootstrap required, opened a new log
I20260929 06:28:51.652052 15605 ts_tablet_manager.cc:1397] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:28:51.652282 15606 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: No bootstrap required, opened a new log
I20260929 06:28:51.652349 15606 ts_tablet_manager.cc:1397] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:51.652971 15609 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Bootstrap starting.
I20260929 06:28:51.653520 15609 tablet_bootstrap.cc:654] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.653815 15609 log.cc:824] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:51.654110 15605 raft_consensus.cc:348] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.654317 15605 raft_consensus.cc:374] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.654351 15605 raft_consensus.cc:729] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.654282 15606 raft_consensus.cc:348] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.654412 15606 raft_consensus.cc:374] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.654450 15606 raft_consensus.cc:729] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.654462 15605 consensus_queue.cc:260] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.654567 15609 tablet_bootstrap.cc:492] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: No bootstrap required, opened a new log
I20260929 06:28:51.654547 15606 consensus_queue.cc:260] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.654637 15609 ts_tablet_manager.cc:1397] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:51.654742 15605 ts_tablet_manager.cc:1428] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 06:28:51.654793 15606 ts_tablet_manager.cc:1428] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:28:51.654908 15302 heartbeater.cc:503] Master 127.13.15.190:44753 was elected leader, sending a full tablet report...
I20260929 06:28:51.655150 15582 heartbeater.cc:503] Master 127.13.15.190:44753 was elected leader, sending a full tablet report...
I20260929 06:28:51.656461 15609 raft_consensus.cc:348] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.656610 15609 raft_consensus.cc:374] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.656651 15609 raft_consensus.cc:729] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.656760 15609 consensus_queue.cc:260] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.656999 15609 ts_tablet_manager.cc:1428] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:28:51.657039 15442 heartbeater.cc:503] Master 127.13.15.190:44753 was elected leader, sending a full tablet report...
W20260929 06:28:51.677563 15443 tablet.cc:2369] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:28:51.746819 15303 tablet.cc:2369] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:28:51.758294 15611 raft_consensus.cc:482] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:51.758406 15611 raft_consensus.cc:504] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.758800 15611 leader_election.cc:288] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647)
I20260929 06:28:51.762043 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.758976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35769 (local address 127.13.15.131:42997)
0929 06:28:51.759014 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.759046 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.759088 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.759309 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.759310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.759312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.759381 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.759382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.760316 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.760317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.761119 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.761121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.761719 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.761799 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.761802 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.761902 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.761964 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":19}
I20260929 06:28:51.762055 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.759005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:42997 (local address 127.13.15.129:35769)
0929 06:28:51.759066 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.759069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.759094 (+ 25us) server_negotiation.cc:407] Connection header received
0929 06:28:51.759196 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.759197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.759211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.759280 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.759454 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.760215 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761202 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.761646 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761701 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.761708 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.761924 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.761987 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:28:51.762157 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.759095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.13.15.131:58799)
0929 06:28:51.759248 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.759256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.759309 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.759431 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.759432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.759433 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.759485 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.759486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.760346 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.760347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.761129 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.761131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.761775 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.761832 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.761834 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.761967 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.762004 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260929 06:28:51.762107 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.759218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:58799 (local address 127.13.15.130:34647)
0929 06:28:51.759269 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.759271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.759297 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:28:51.759362 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.759363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.759369 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.759409 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.759552 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.760244 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761231 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.761669 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.761759 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.761768 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.761994 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.762038 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:28:51.762425 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fa09b5529a4b349274be3adf915aee" candidate_uuid: "85d214e3f6374a30bd6721d5951a68bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1" is_pre_election: true
I20260929 06:28:51.762575 15256 raft_consensus.cc:2454] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d214e3f6374a30bd6721d5951a68bd in term 0.
I20260929 06:28:51.762699 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fa09b5529a4b349274be3adf915aee" candidate_uuid: "85d214e3f6374a30bd6721d5951a68bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c" is_pre_election: true
I20260929 06:28:51.762773 15470 leader_election.cc:302] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d214e3f6374a30bd6721d5951a68bd, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:51.762853 15396 raft_consensus.cc:2454] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d214e3f6374a30bd6721d5951a68bd in term 0.
I20260929 06:28:51.762917 15611 raft_consensus.cc:2781] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:51.762987 15611 raft_consensus.cc:482] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:51.763011 15611 raft_consensus.cc:3037] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.763708 15611 raft_consensus.cc:504] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.763844 15611 leader_election.cc:288] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [CANDIDATE]: Term 1 election: Requested vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647)
I20260929 06:28:51.763973 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fa09b5529a4b349274be3adf915aee" candidate_uuid: "85d214e3f6374a30bd6721d5951a68bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1"
I20260929 06:28:51.764012 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fa09b5529a4b349274be3adf915aee" candidate_uuid: "85d214e3f6374a30bd6721d5951a68bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c"
I20260929 06:28:51.764045 15256 raft_consensus.cc:3037] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.764108 15396 raft_consensus.cc:3037] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.764747 15256 raft_consensus.cc:2454] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d214e3f6374a30bd6721d5951a68bd in term 1.
I20260929 06:28:51.764894 15470 leader_election.cc:302] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d214e3f6374a30bd6721d5951a68bd, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:51.764897 15396 raft_consensus.cc:2454] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d214e3f6374a30bd6721d5951a68bd in term 1.
I20260929 06:28:51.764985 15611 raft_consensus.cc:2781] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:51.765144 15611 raft_consensus.cc:686] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 1 LEADER]: Becoming Leader. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Running, Role: LEADER
I20260929 06:28:51.765247 15611 consensus_queue.cc:237] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } }
I20260929 06:28:51.766045 15098 catalog_manager.cc:5495] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131). New cstate: current_term: 1 leader_uuid: "85d214e3f6374a30bd6721d5951a68bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } health_report { overall_health: UNKNOWN } } }
Sep 29 06:28:51 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:51.828152 15622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.814938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44753 (local address 127.0.0.1:53214)
0929 06:28:51.815084 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.815095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.815175 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.815367 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.815368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.815705 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:51.815752 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.815755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.816427 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.816428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.816973 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.816974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.817362 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.817363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.817415 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.824371 (+ 6956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:51.824376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:51.825841 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:51.827241 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.827247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.827249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.827440 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.827606 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:51.827607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:51.827607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:51.827657 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:51.827875 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:51.827880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:51.827920 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.827970 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.828061 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260929 06:28:51.828270 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.814961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.13.15.190:44753)
0929 06:28:51.815042 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.815045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.815159 (+ 114us) server_negotiation.cc:407] Connection header received
0929 06:28:51.815251 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.815252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.815271 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.815300 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:51.815855 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.816345 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.817079 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.817276 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.817361 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.825940 (+ 8579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:51.825941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:51.825942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:51.825948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:51.827167 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.827503 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.827504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.827504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.827551 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:51.827707 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:51.827708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:51.827708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:51.827809 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:51.827857 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.827999 (+ 142us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.828161 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":38}
I20260929 06:28:51.830267 15098 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:53214:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 06:28:51.830433 15098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:51.831887 15516 tablet_service.cc:1467] Processing CreateTablet for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d3d9dfbe882646009c2c6aec4ded6388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:51.831880 15236 tablet_service.cc:1467] Processing CreateTablet for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d3d9dfbe882646009c2c6aec4ded6388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:51.831925 15376 tablet_service.cc:1467] Processing CreateTablet for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d3d9dfbe882646009c2c6aec4ded6388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:28:51.832062 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.832062 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.832062 15516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8be92f57d94d8c8b0d088e3ca9fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.833559 15606 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: Bootstrap starting.
I20260929 06:28:51.833564 15605 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: Bootstrap starting.
I20260929 06:28:51.833920 15606 tablet_bootstrap.cc:654] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.833982 15605 tablet_bootstrap.cc:654] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.834401 15609 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: Bootstrap starting.
I20260929 06:28:51.834568 15606 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: No bootstrap required, opened a new log
I20260929 06:28:51.834647 15606 ts_tablet_manager.cc:1397] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:28:51.834766 15605 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: No bootstrap required, opened a new log
I20260929 06:28:51.834811 15605 ts_tablet_manager.cc:1397] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:51.834820 15606 raft_consensus.cc:348] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.834920 15606 raft_consensus.cc:374] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.834940 15606 raft_consensus.cc:729] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.834988 15606 consensus_queue.cc:260] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.834973 15605 raft_consensus.cc:348] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.835042 15605 raft_consensus.cc:374] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.835063 15605 raft_consensus.cc:729] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.835090 15609 tablet_bootstrap.cc:654] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.835100 15606 ts_tablet_manager.cc:1428] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:51.835114 15605 consensus_queue.cc:260] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.835207 15605 ts_tablet_manager.cc:1428] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.835240 15612 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.835325 15612 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.835530 15611 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.835618 15611 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.835933 15609 tablet_bootstrap.cc:492] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: No bootstrap required, opened a new log
I20260929 06:28:51.835983 15609 ts_tablet_manager.cc:1397] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:51.836125 15609 raft_consensus.cc:348] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.836207 15609 raft_consensus.cc:374] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.836238 15609 raft_consensus.cc:729] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.836287 15609 consensus_queue.cc:260] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.836407 15609 ts_tablet_manager.cc:1428] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.836490 15613 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.836621 15613 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: This TxnStatusTablet replica's current role is: FOLLOWER
W20260929 06:28:51.859298 15583 tablet.cc:2369] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:28:51.902621 15613 raft_consensus.cc:482] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:51.902716 15613 raft_consensus.cc:504] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.903048 15613 leader_election.cc:288] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:51.905844 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.903218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35149 (local address 127.13.15.130:33053)
0929 06:28:51.903278 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.903290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.903338 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.903537 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.903538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.903540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.903618 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.903619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.904268 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.904269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.905020 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.905023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.905548 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.905637 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.905640 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.905691 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.905719 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 06:28:51.905845 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.903357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33053 (local address 127.13.15.131:35149)
0929 06:28:51.903411 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.903413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.903424 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:28:51.903453 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.903454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.903463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.903522 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.903693 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.904187 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905107 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.905469 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905523 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.905530 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.905715 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.905769 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:28:51.905884 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.903347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:54087 (local address 127.13.15.129:35769)
0929 06:28:51.903393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.903395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.903510 (+ 115us) server_negotiation.cc:407] Connection header received
0929 06:28:51.903539 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.903540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.903548 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.903607 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:51.903779 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.904234 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905142 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.905529 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905600 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.905607 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.905772 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.905814 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:28:51.905943 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.903244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35769 (local address 127.13.15.130:54087)
0929 06:28:51.903405 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.903425 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.903489 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.903617 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.903618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.903619 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:51.903707 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.903709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.904313 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.904314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.905029 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.905031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.905622 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.905623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.905669 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.905672 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.905748 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.905782 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260929 06:28:51.906188 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8be92f57d94d8c8b0d088e3ca9fb7f" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1" is_pre_election: true
I20260929 06:28:51.906198 15536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8be92f57d94d8c8b0d088e3ca9fb7f" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd" is_pre_election: true
I20260929 06:28:51.906280 15256 raft_consensus.cc:2454] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 0.
I20260929 06:28:51.906311 15536 raft_consensus.cc:2454] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 0.
I20260929 06:28:51.906474 15330 leader_election.cc:302] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:51.906607 15613 raft_consensus.cc:2781] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:51.906659 15613 raft_consensus.cc:482] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:51.906687 15613 raft_consensus.cc:3037] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.907359 15613 raft_consensus.cc:504] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.907514 15613 leader_election.cc:288] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 election: Requested vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:51.907683 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8be92f57d94d8c8b0d088e3ca9fb7f" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1"
I20260929 06:28:51.907714 15536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8be92f57d94d8c8b0d088e3ca9fb7f" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd"
I20260929 06:28:51.907773 15256 raft_consensus.cc:3037] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.907778 15536 raft_consensus.cc:3037] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:51.908370 15256 raft_consensus.cc:2454] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 1.
I20260929 06:28:51.908433 15536 raft_consensus.cc:2454] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 1.
I20260929 06:28:51.908576 15330 leader_election.cc:302] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:51.908703 15613 raft_consensus.cc:2781] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:51.908926 15613 raft_consensus.cc:686] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [term 1 LEADER]: Becoming Leader. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Running, Role: LEADER
I20260929 06:28:51.909029 15613 consensus_queue.cc:237] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.909502 15627 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: TxnStatusTablet state changed. Reason: New leader 7c3ac9cbf8314d20863537e39646a80c. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.909567 15627 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:51.909806 15098 catalog_manager.cc:5495] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130). New cstate: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:51.910136 15629 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 06:28:51.910282 15629 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 06:28:51.930732 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.927635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.13.15.130:34647)
0929 06:28:51.927703 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.927705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.927714 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:28:51.927796 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.927797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.927818 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.927902 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:51.928085 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.928562 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.929716 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.930051 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.930200 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.930388 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.930492 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.930559 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.930625 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.930646 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:28:51.930765 15622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.927557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.0.0.1:47650)
0929 06:28:51.927637 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.927647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.927710 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.927942 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.927943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.927945 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:51.927999 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.928002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.928637 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.928638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.929607 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.929610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.930187 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.930188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.930260 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.930279 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.930564 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.930571 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.930621 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.930670 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 06:28:51.932859 15256 raft_consensus.cc:1261] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Refusing update from remote peer 7c3ac9cbf8314d20863537e39646a80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:51.932874 15536 raft_consensus.cc:1261] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Refusing update from remote peer 7c3ac9cbf8314d20863537e39646a80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:51.933187 15627 consensus_queue.cc:1035] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:51.933341 15613 consensus_queue.cc:1035] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:51.934523 15611 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: TxnStatusTablet state changed. Reason: New leader 7c3ac9cbf8314d20863537e39646a80c. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.934609 15611 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.934556 15612 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: TxnStatusTablet state changed. Reason: New leader 7c3ac9cbf8314d20863537e39646a80c. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.934667 15612 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.935521 15611 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.935593 15611 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 85d214e3f6374a30bd6721d5951a68bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.935545 15627 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.935637 15627 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:51.936547 15613 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.936627 15613 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 7c3ac9cbf8314d20863537e39646a80c: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:28:51.937340 15634 mvcc.cc:205] Tried to move back new op lower bound from 7334557007594352640 to 7334557007499862016. Current Snapshot: MvccSnapshot[applied={T|T < 7334557007594352640}]
I20260929 06:28:51.937455 15633 mvcc.cc:205] Tried to move back new op lower bound from 7334557007594352640 to 7334557007499862016. Current Snapshot: MvccSnapshot[applied={T|T < 7334557007594352640}]
I20260929 06:28:51.937419 15612 tablet_replica.cc:439] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } } }
I20260929 06:28:51.937507 15612 tablet_replica.cc:442] T ce8be92f57d94d8c8b0d088e3ca9fb7f P 93fa3708bf6b4a89878b78d478d49bb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:28:51.941243 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.938387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.13.15.131:35149)
0929 06:28:51.938428 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.938431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.938438 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:28:51.938471 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.938471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.938489 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.938564 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:51.938749 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.939513 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.940414 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.940704 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.940780 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.940904 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.941012 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.941107 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.941156 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.941176 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:28:51.946224 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.943678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:35195 (local address 127.13.15.130:34647)
0929 06:28:51.943723 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.943725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.943761 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:28:51.943844 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.943845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.943854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.943914 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:51.944063 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.944568 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.945415 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.945627 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.945724 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.945922 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.945962 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.946027 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.946095 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.946121 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:28:51.946231 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.943606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.13.15.131:35195)
0929 06:28:51.943712 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.943725 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.943777 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.943916 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.943917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.943918 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:51.943973 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.943975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.944658 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.944659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.945322 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.945324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.945722 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.945723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.945793 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.945811 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.946029 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.946031 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.946090 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.946152 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 06:28:51.948947 15396 raft_consensus.cc:1261] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Refusing update from remote peer 85d214e3f6374a30bd6721d5951a68bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:51.949132 15256 raft_consensus.cc:1261] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Refusing update from remote peer 85d214e3f6374a30bd6721d5951a68bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:51.949229 15611 consensus_queue.cc:1035] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:51.949347 15615 consensus_queue.cc:1035] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:51.951923 15632 mvcc.cc:205] Tried to move back new op lower bound from 7334557007660724224 to 7334557006911090688. Current Snapshot: MvccSnapshot[applied={T|T < 7334557007660724224}]
I20260929 06:28:51.958573 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.954934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35149 (local address 127.13.15.130:33613)
0929 06:28:51.955097 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:51.955109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:51.955173 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:51.955285 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:51.955286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:51.955287 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:51.955341 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.955342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.956171 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.956171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.957727 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:51.957729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.958201 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.958202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:51.958269 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.958281 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.958445 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.958446 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:28:51.958479 (+ 33us) client_negotiation.cc:240] Negotiation successful
0929 06:28:51.958511 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260929 06:28:51.958573 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:51.954986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33613 (local address 127.13.15.131:35149)
0929 06:28:51.955019 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:51.955021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:51.955180 (+ 159us) server_negotiation.cc:407] Connection header received
0929 06:28:51.955208 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:51.955209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:51.955218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:51.955283 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:51.955396 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.956094 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.957806 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:51.958088 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:51.958190 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:51.958354 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:51.958384 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:51.958436 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:51.958490 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 06:28:51.958510 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:28:51.966677 15099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:51.966782 15099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:51.967536 15099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Sending DeleteTablet for 3 replicas of tablet 31fa09b5529a4b349274be3adf915aee
I20260929 06:28:51.967880 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fa09b5529a4b349274be3adf915aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58518
I20260929 06:28:51.967944 15376 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fa09b5529a4b349274be3adf915aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:47636
I20260929 06:28:51.968251 15516 tablet_service.cc:1514] Processing DeleteTablet for tablet 31fa09b5529a4b349274be3adf915aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57828
I20260929 06:28:51.968544 15653 tablet_replica.cc:330] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: stopping tablet replica
I20260929 06:28:51.968652 15653 raft_consensus.cc:2229] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:28:51.968713 15099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 06:28:51.968791 15653 raft_consensus.cc:2258] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 06:28:51.968848 15099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:51.969550 15653 ts_tablet_manager.cc:1905] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:51.970208 15516 tablet_service.cc:1467] Processing CreateTablet for tablet df491ab15b384592b514d902ec20808d (DEFAULT_TABLE table=test-table [id=e538db9408c343188bf2376bed33bee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.970345 15516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df491ab15b384592b514d902ec20808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.971726 15376 tablet_service.cc:1467] Processing CreateTablet for tablet df491ab15b384592b514d902ec20808d (DEFAULT_TABLE table=test-table [id=e538db9408c343188bf2376bed33bee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.971756 15605 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Bootstrap starting.
I20260929 06:28:51.971834 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df491ab15b384592b514d902ec20808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.972163 15605 tablet_bootstrap.cc:654] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.973119 15609 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Bootstrap starting.
I20260929 06:28:51.973501 15609 tablet_bootstrap.cc:654] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.974086 15236 tablet_service.cc:1467] Processing CreateTablet for tablet df491ab15b384592b514d902ec20808d (DEFAULT_TABLE table=test-table [id=e538db9408c343188bf2376bed33bee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:51.974165 15609 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: No bootstrap required, opened a new log
I20260929 06:28:51.974205 15609 ts_tablet_manager.cc:1397] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:51.974251 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df491ab15b384592b514d902ec20808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:51.974350 15609 raft_consensus.cc:348] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.974408 15609 raft_consensus.cc:374] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.974428 15609 raft_consensus.cc:729] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.974465 15609 consensus_queue.cc:260] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.974539 15609 ts_tablet_manager.cc:1428] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.975409 15651 tablet_replica.cc:330] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: stopping tablet replica
I20260929 06:28:51.975524 15651 raft_consensus.cc:2229] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:51.975589 15651 raft_consensus.cc:2258] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:51.975790 15606 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Bootstrap starting.
I20260929 06:28:51.976132 15606 tablet_bootstrap.cc:654] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:51.977176 15652 tablet_replica.cc:330] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: stopping tablet replica
I20260929 06:28:51.977279 15652 raft_consensus.cc:2229] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:51.977326 15652 raft_consensus.cc:2258] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:51.977458 15653 ts_tablet_manager.cc:1918] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:51.977553 15653 log.cc:1196] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/31fa09b5529a4b349274be3adf915aee
I20260929 06:28:51.977830 15652 ts_tablet_manager.cc:1905] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:51.977857 15653 ts_tablet_manager.cc:1939] T 31fa09b5529a4b349274be3adf915aee P 85d214e3f6374a30bd6721d5951a68bd: Deleting consensus metadata
I20260929 06:28:51.978372 15083 catalog_manager.cc:4841] TS 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149): tablet 31fa09b5529a4b349274be3adf915aee (table test-table [id=37eb1dd994fa45dcb8805906168c8b84]) successfully deleted
I20260929 06:28:51.978475 15605 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: No bootstrap required, opened a new log
I20260929 06:28:51.978523 15605 ts_tablet_manager.cc:1397] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260929 06:28:51.978665 15605 raft_consensus.cc:348] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.978739 15605 raft_consensus.cc:374] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.978772 15605 raft_consensus.cc:729] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.978817 15605 consensus_queue.cc:260] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.978901 15605 ts_tablet_manager.cc:1428] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:51.979521 15652 ts_tablet_manager.cc:1918] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:51.979602 15652 log.cc:1196] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/31fa09b5529a4b349274be3adf915aee
I20260929 06:28:51.979812 15651 ts_tablet_manager.cc:1905] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:51.979892 15652 ts_tablet_manager.cc:1939] T 31fa09b5529a4b349274be3adf915aee P 7c3ac9cbf8314d20863537e39646a80c: Deleting consensus metadata
I20260929 06:28:51.980288 15086 catalog_manager.cc:4841] TS 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647): tablet 31fa09b5529a4b349274be3adf915aee (table test-table [id=37eb1dd994fa45dcb8805906168c8b84]) successfully deleted
I20260929 06:28:51.981334 15651 ts_tablet_manager.cc:1918] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:51.981401 15651 log.cc:1196] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/31fa09b5529a4b349274be3adf915aee
I20260929 06:28:51.981617 15651 ts_tablet_manager.cc:1939] T 31fa09b5529a4b349274be3adf915aee P 93fa3708bf6b4a89878b78d478d49bb1: Deleting consensus metadata
I20260929 06:28:51.981995 15086 catalog_manager.cc:4841] TS 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769): tablet 31fa09b5529a4b349274be3adf915aee (table test-table [id=37eb1dd994fa45dcb8805906168c8b84]) successfully deleted
I20260929 06:28:51.986575 15606 tablet_bootstrap.cc:492] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: No bootstrap required, opened a new log
I20260929 06:28:51.986646 15606 ts_tablet_manager.cc:1397] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260929 06:28:51.986791 15606 raft_consensus.cc:348] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.986867 15606 raft_consensus.cc:374] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:51.986889 15606 raft_consensus.cc:729] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Initialized, Role: FOLLOWER
I20260929 06:28:51.986941 15606 consensus_queue.cc:260] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:51.987032 15606 ts_tablet_manager.cc:1428] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:52.226608 15627 raft_consensus.cc:482] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:52.226698 15627 raft_consensus.cc:504] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:52.226866 15627 leader_election.cc:288] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:52.227073 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df491ab15b384592b514d902ec20808d" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1" is_pre_election: true
I20260929 06:28:52.227157 15256 raft_consensus.cc:2454] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 0.
I20260929 06:28:52.227100 15536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df491ab15b384592b514d902ec20808d" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd" is_pre_election: true
I20260929 06:28:52.227240 15536 raft_consensus.cc:2454] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 0.
I20260929 06:28:52.227319 15330 leader_election.cc:302] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:52.227423 15627 raft_consensus.cc:2781] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:52.227473 15627 raft_consensus.cc:482] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:52.227504 15627 raft_consensus.cc:3037] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:52.228248 15627 raft_consensus.cc:504] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:52.228411 15627 leader_election.cc:288] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 election: Requested vote from peers 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:52.228528 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df491ab15b384592b514d902ec20808d" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa3708bf6b4a89878b78d478d49bb1"
I20260929 06:28:52.228562 15536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df491ab15b384592b514d902ec20808d" candidate_uuid: "7c3ac9cbf8314d20863537e39646a80c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd"
I20260929 06:28:52.228602 15256 raft_consensus.cc:3037] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:52.228638 15536 raft_consensus.cc:3037] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:52.229283 15256 raft_consensus.cc:2454] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 1.
I20260929 06:28:52.229323 15536 raft_consensus.cc:2454] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3ac9cbf8314d20863537e39646a80c in term 1.
I20260929 06:28:52.229477 15330 leader_election.cc:302] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:52.229571 15627 raft_consensus.cc:2781] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:52.229638 15627 raft_consensus.cc:686] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 1 LEADER]: Becoming Leader. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Running, Role: LEADER
I20260929 06:28:52.229714 15627 consensus_queue.cc:237] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:52.230352 15099 catalog_manager.cc:5495] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130). New cstate: current_term: 1 leader_uuid: "7c3ac9cbf8314d20863537e39646a80c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:52.258431 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:52.256581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.13.15.130:34647)
0929 06:28:52.256638 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:52.256639 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:52.256646 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:28:52.256663 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:52.256664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:52.256672 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:52.256715 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:52.256833 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.257260 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.257885 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.258047 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.258092 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:52.258259 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:52.258284 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:52.258335 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:52.258372 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 06:28:52.258385 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 06:28:52.262341 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:52.260300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33889 (local address 127.13.15.130:34647)
0929 06:28:52.260341 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:52.260342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:52.260350 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:28:52.260405 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:52.260405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:52.260413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:52.260460 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:52.260584 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.261020 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.261687 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.261898 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.261964 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:52.262088 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:52.262113 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:52.262197 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:52.262247 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 06:28:52.262266 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":25}
I20260929 06:28:52.262444 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:52.260244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.13.15.130:33889)
0929 06:28:52.260302 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:52.260312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:52.260351 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:52.260461 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:52.260461 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:52.260462 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:52.260496 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:52.260497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.261097 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.261098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:52.261605 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:52.261606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:52.261974 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:52.261974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:52.262021 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:52.262024 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:52.262201 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:52.262202 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:28:52.262246 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 06:28:52.262298 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 06:28:52.264041 15256 raft_consensus.cc:1261] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Refusing update from remote peer 7c3ac9cbf8314d20863537e39646a80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:52.264041 15536 raft_consensus.cc:1261] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Refusing update from remote peer 7c3ac9cbf8314d20863537e39646a80c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:52.264245 15613 consensus_queue.cc:1035] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:52.264374 15627 consensus_queue.cc:1035] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 06:28:52.267859 15632 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 06:28:53.785518 15099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:53.785625 15099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:53.786412 15099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Sending DeleteTablet for 3 replicas of tablet df491ab15b384592b514d902ec20808d
I20260929 06:28:53.786659 15516 tablet_service.cc:1514] Processing DeleteTablet for tablet df491ab15b384592b514d902ec20808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:53 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57828
I20260929 06:28:53.786648 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet df491ab15b384592b514d902ec20808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:53 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58518
I20260929 06:28:53.786684 15376 tablet_service.cc:1514] Processing DeleteTablet for tablet df491ab15b384592b514d902ec20808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:53 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:47636
I20260929 06:28:53.786855 15683 tablet_replica.cc:330] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: stopping tablet replica
I20260929 06:28:53.786928 15683 raft_consensus.cc:2229] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:53.786963 15683 raft_consensus.cc:2258] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:53.786864 15684 tablet_replica.cc:330] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: stopping tablet replica
I20260929 06:28:53.787169 15683 ts_tablet_manager.cc:1905] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:53.787200 15684 raft_consensus.cc:2229] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:28:53.787288 15684 raft_consensus.cc:2258] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:53.787503 15684 ts_tablet_manager.cc:1905] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:53.788681 15682 tablet_replica.cc:330] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: stopping tablet replica
I20260929 06:28:53.788738 15682 raft_consensus.cc:2229] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:53.788772 15682 raft_consensus.cc:2258] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:53.789443 15683 ts_tablet_manager.cc:1918] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 06:28:53.789511 15683 log.cc:1196] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/df491ab15b384592b514d902ec20808d
I20260929 06:28:53.789492 15684 ts_tablet_manager.cc:1918] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 06:28:53.789577 15684 log.cc:1196] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/df491ab15b384592b514d902ec20808d
I20260929 06:28:53.789685 15682 ts_tablet_manager.cc:1905] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:53.789762 15683 ts_tablet_manager.cc:1939] T df491ab15b384592b514d902ec20808d P 93fa3708bf6b4a89878b78d478d49bb1: Deleting consensus metadata
I20260929 06:28:53.789934 15684 ts_tablet_manager.cc:1939] T df491ab15b384592b514d902ec20808d P 7c3ac9cbf8314d20863537e39646a80c: Deleting consensus metadata
I20260929 06:28:53.790081 15086 catalog_manager.cc:4841] TS 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769): tablet df491ab15b384592b514d902ec20808d (table test-table [id=e538db9408c343188bf2376bed33bee8]) successfully deleted
I20260929 06:28:53.790277 15086 catalog_manager.cc:4841] TS 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647): tablet df491ab15b384592b514d902ec20808d (table test-table [id=e538db9408c343188bf2376bed33bee8]) successfully deleted
I20260929 06:28:53.791036 15682 ts_tablet_manager.cc:1918] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 06:28:53.791093 15682 log.cc:1196] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/df491ab15b384592b514d902ec20808d
I20260929 06:28:53.791308 15682 ts_tablet_manager.cc:1939] T df491ab15b384592b514d902ec20808d P 85d214e3f6374a30bd6721d5951a68bd: Deleting consensus metadata
I20260929 06:28:53.791607 15083 catalog_manager.cc:4841] TS 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149): tablet df491ab15b384592b514d902ec20808d (table test-table [id=e538db9408c343188bf2376bed33bee8]) successfully deleted
Sep 29 06:28:53 dist-test-slave-sm0p krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 06:28:53 dist-test-slave-sm0p krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:53.810967 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:53.802872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.13.15.190:44753)
0929 06:28:53.802998 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:53.803000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:53.803125 (+ 125us) server_negotiation.cc:407] Connection header received
0929 06:28:53.803205 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:53.803205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:53.803217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:53.803293 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:53.803869 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.804378 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.805125 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.805328 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.805424 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:53.807812 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:53.807813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:53.807814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:53.807821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:53.809475 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:53.809948 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:53.809949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:53.809950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:53.810014 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:53.810324 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:53.810325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:53.810325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:53.810484 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:53.810587 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:53.810754 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 06:28:53.810875 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260929 06:28:53.812968 15099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:28:53.813145 15099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:53.814461 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 (DEFAULT_TABLE table=test-table [id=30a089c06d4a4ec69f17884191a198f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:53.814522 15376 tablet_service.cc:1467] Processing CreateTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 (DEFAULT_TABLE table=test-table [id=30a089c06d4a4ec69f17884191a198f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:53.814610 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813377c3f5043c9a926e4b6e5c72669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:53.814651 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813377c3f5043c9a926e4b6e5c72669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:53.815582 15693 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Bootstrap starting.
I20260929 06:28:53.815649 15516 tablet_service.cc:1467] Processing CreateTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 (DEFAULT_TABLE table=test-table [id=30a089c06d4a4ec69f17884191a198f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:53.815770 15516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813377c3f5043c9a926e4b6e5c72669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:53.815920 15693 tablet_bootstrap.cc:654] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:53.816473 15693 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: No bootstrap required, opened a new log
I20260929 06:28:53.816527 15693 ts_tablet_manager.cc:1397] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:53.816658 15693 raft_consensus.cc:348] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.816725 15693 raft_consensus.cc:374] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:53.816742 15693 raft_consensus.cc:729] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Initialized, Role: FOLLOWER
I20260929 06:28:53.816804 15693 consensus_queue.cc:260] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.816948 15693 ts_tablet_manager.cc:1428] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:53.817114 15697 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Bootstrap starting.
I20260929 06:28:53.817284 15694 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Bootstrap starting.
I20260929 06:28:53.817597 15697 tablet_bootstrap.cc:654] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:53.817651 15694 tablet_bootstrap.cc:654] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:53.818238 15697 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: No bootstrap required, opened a new log
I20260929 06:28:53.818282 15697 ts_tablet_manager.cc:1397] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:53.818384 15697 raft_consensus.cc:348] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.818462 15697 raft_consensus.cc:374] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:53.818476 15697 raft_consensus.cc:729] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Initialized, Role: FOLLOWER
I20260929 06:28:53.818526 15697 consensus_queue.cc:260] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.818684 15697 ts_tablet_manager.cc:1428] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:53.818902 15694 tablet_bootstrap.cc:492] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: No bootstrap required, opened a new log
I20260929 06:28:53.818948 15694 ts_tablet_manager.cc:1397] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:53.819065 15694 raft_consensus.cc:348] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.819130 15694 raft_consensus.cc:374] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:53.819147 15694 raft_consensus.cc:729] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Initialized, Role: FOLLOWER
I20260929 06:28:53.819203 15694 consensus_queue.cc:260] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.819321 15694 ts_tablet_manager.cc:1428] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:53.972369 15696 raft_consensus.cc:482] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:53.972463 15696 raft_consensus.cc:504] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.972857 15696 leader_election.cc:288] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:53.975948 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:53.973055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.13.15.129:40171)
0929 06:28:53.973286 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:53.973297 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:53.973349 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:53.973566 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:53.973566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:53.973568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:53.973648 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:53.973650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.974362 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.974363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:53.975123 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:53.975125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.975681 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:53.975756 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:53.975759 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:53.975823 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 06:28:53.975863 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":141,"threads_started":1}
I20260929 06:28:53.975941 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:53.973109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:40171 (local address 127.13.15.130:34647)
0929 06:28:53.973441 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:53.973442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:53.973449 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:28:53.973471 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:53.973471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:53.973479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:53.973554 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:53.973724 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.974278 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975220 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.975593 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975682 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:53.975691 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:53.975836 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 06:28:53.975872 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":92,"threads_started":1}
I20260929 06:28:53.976135 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:53.973066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35149 (local address 127.13.15.129:58291)
0929 06:28:53.973343 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:53.973351 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:53.973431 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:53.973594 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:53.973595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:53.973597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:28:53.973905 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:53.973907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.974571 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.974572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:53.975313 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:53.975315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.975874 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:53.975946 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:53.975949 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:28:53.976011 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 06:28:53.976052 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":138,"threads_started":1}
I20260929 06:28:53.976135 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:53.973128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:58291 (local address 127.13.15.131:35149)
0929 06:28:53.973289 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:53.973291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:53.973419 (+ 128us) server_negotiation.cc:407] Connection header received
0929 06:28:53.973491 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:53.973491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:53.973502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:53.973590 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:28:53.973989 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.974485 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975408 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:53.975783 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:53.975872 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:53.975879 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:53.976029 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 06:28:53.976071 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260929 06:28:53.976261 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813377c3f5043c9a926e4b6e5c72669" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c" is_pre_election: true
I20260929 06:28:53.976354 15396 raft_consensus.cc:2454] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 0.
I20260929 06:28:53.976418 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813377c3f5043c9a926e4b6e5c72669" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd" is_pre_election: true
I20260929 06:28:53.976511 15535 raft_consensus.cc:2454] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 0.
I20260929 06:28:53.976569 15190 leader_election.cc:302] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:53.976734 15696 raft_consensus.cc:2781] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:53.976783 15696 raft_consensus.cc:482] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:53.976799 15696 raft_consensus.cc:3037] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:53.977449 15696 raft_consensus.cc:504] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.977581 15696 leader_election.cc:288] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 election: Requested vote from peers 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:53.977726 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813377c3f5043c9a926e4b6e5c72669" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd"
I20260929 06:28:53.977738 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813377c3f5043c9a926e4b6e5c72669" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c"
I20260929 06:28:53.977809 15535 raft_consensus.cc:3037] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:53.977833 15396 raft_consensus.cc:3037] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:53.978489 15535 raft_consensus.cc:2454] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 1.
I20260929 06:28:53.978556 15396 raft_consensus.cc:2454] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 1.
I20260929 06:28:53.978726 15187 leader_election.cc:302] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d214e3f6374a30bd6721d5951a68bd, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:53.978857 15696 raft_consensus.cc:2781] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:53.979024 15696 raft_consensus.cc:686] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Running, Role: LEADER
I20260929 06:28:53.979100 15696 consensus_queue.cc:237] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:53.979808 15098 catalog_manager.cc:5495] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "93fa3708bf6b4a89878b78d478d49bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:54.002643 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:54.000274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.13.15.129:35769)
0929 06:28:54.000435 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:54.000437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:54.000444 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:28:54.000481 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:54.000482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:54.000498 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:54.000564 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:54.000643 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.001164 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.001911 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.002173 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.002246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:54.002369 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:54.002467 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:54.002517 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:54.002555 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 06:28:54.002568 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260929 06:28:54.006933 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:54.004894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56263 (local address 127.13.15.130:34647)
0929 06:28:54.004963 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:54.004965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:54.004992 (+ 27us) server_negotiation.cc:407] Connection header received
0929 06:28:54.005033 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:54.005034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:54.005042 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:54.005086 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:54.005187 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.005640 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.006401 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.006577 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.006639 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:54.006763 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:54.006784 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:54.006819 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:54.006871 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 06:28:54.006886 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 06:28:54.006994 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:54.004746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.13.15.129:56263)
0929 06:28:54.004887 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:54.004897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:54.004958 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:54.005096 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:54.005096 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:54.005097 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:54.005138 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:54.005139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.005696 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.005696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:54.006338 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:54.006340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.006647 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.006648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:54.006692 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:54.006702 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:54.006836 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:54.006838 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:28:54.006870 (+ 32us) client_negotiation.cc:240] Negotiation successful
0929 06:28:54.006900 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260929 06:28:54.008812 15396 raft_consensus.cc:1261] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Refusing update from remote peer 93fa3708bf6b4a89878b78d478d49bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:54.008805 15535 raft_consensus.cc:1261] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Refusing update from remote peer 93fa3708bf6b4a89878b78d478d49bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:54.009073 15705 consensus_queue.cc:1035] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:54.009197 15696 consensus_queue.cc:1035] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:54.015925 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:54.013814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:42615 (local address 127.13.15.129:35769)
0929 06:28:54.013849 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:54.013851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:54.013956 (+ 105us) server_negotiation.cc:407] Connection header received
0929 06:28:54.014007 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:54.014008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:54.014015 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:54.014049 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:28:54.014124 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.014594 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.015262 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.015554 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.015611 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:54.015744 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:28:54.015771 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:28:54.015824 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:54.015873 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 06:28:54.015887 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:28:54.015954 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:54.013780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35769 (local address 127.13.15.130:42615)
0929 06:28:54.013896 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:54.013905 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:54.013962 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:54.014052 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:54.014052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:54.014053 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:28:54.014087 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:54.014088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.014653 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.014653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:54.015181 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:54.015182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:54.015620 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:54.015620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:54.015676 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:54.015680 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:28:54.015832 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:28:54.015833 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:28:54.015873 (+ 40us) client_negotiation.cc:240] Negotiation successful
0929 06:28:54.015905 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":78,"threads_started":1}
I20260929 06:28:54.022313 15098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:54.022394 15098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:54.022994 15098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Sending DeleteTablet for 3 replicas of tablet 5813377c3f5043c9a926e4b6e5c72669
I20260929 06:28:54.023200 15516 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:54 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57828
I20260929 06:28:54.023254 15376 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:54 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:47636
I20260929 06:28:54.023286 15682 tablet_replica.cc:330] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: stopping tablet replica
I20260929 06:28:54.023301 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813377c3f5043c9a926e4b6e5c72669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:54 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58518
I20260929 06:28:54.023344 15682 raft_consensus.cc:2229] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:54.023377 15682 raft_consensus.cc:2258] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:54.023396 15683 tablet_replica.cc:330] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: stopping tablet replica
I20260929 06:28:54.023451 15683 raft_consensus.cc:2229] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:28:54.023448 15684 tablet_replica.cc:330] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: stopping tablet replica
I20260929 06:28:54.023494 15684 raft_consensus.cc:2229] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:54.023520 15684 raft_consensus.cc:2258] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:54.023527 15683 raft_consensus.cc:2258] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:54.023660 15098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:28:54.023805 15098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:28:54.023835 15683 ts_tablet_manager.cc:1905] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:54.024029 15682 ts_tablet_manager.cc:1905] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:54.025542 15683 ts_tablet_manager.cc:1918] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:54.025600 15683 log.cc:1196] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/5813377c3f5043c9a926e4b6e5c72669
I20260929 06:28:54.025599 15682 ts_tablet_manager.cc:1918] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:54.025650 15682 log.cc:1196] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/5813377c3f5043c9a926e4b6e5c72669
I20260929 06:28:54.025660 15684 ts_tablet_manager.cc:1905] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:54.025862 15683 ts_tablet_manager.cc:1939] T 5813377c3f5043c9a926e4b6e5c72669 P 93fa3708bf6b4a89878b78d478d49bb1: Deleting consensus metadata
I20260929 06:28:54.025870 15682 ts_tablet_manager.cc:1939] T 5813377c3f5043c9a926e4b6e5c72669 P 85d214e3f6374a30bd6721d5951a68bd: Deleting consensus metadata
I20260929 06:28:54.026162 15086 catalog_manager.cc:4841] TS 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769): tablet 5813377c3f5043c9a926e4b6e5c72669 (table test-table [id=30a089c06d4a4ec69f17884191a198f8]) successfully deleted
I20260929 06:28:54.026474 15083 catalog_manager.cc:4841] TS 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149): tablet 5813377c3f5043c9a926e4b6e5c72669 (table test-table [id=30a089c06d4a4ec69f17884191a198f8]) successfully deleted
I20260929 06:28:54.026811 15516 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 (DEFAULT_TABLE table=test-table [id=bd921a454fd7431b81621edb5c743483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:54.026845 15236 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 (DEFAULT_TABLE table=test-table [id=bd921a454fd7431b81621edb5c743483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:54.026858 15376 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 (DEFAULT_TABLE table=test-table [id=bd921a454fd7431b81621edb5c743483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:28:54.026895 15516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf702c9db6a49d08746678fd6bb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:54.026937 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf702c9db6a49d08746678fd6bb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:54.026949 15236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf702c9db6a49d08746678fd6bb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:54.027207 15684 ts_tablet_manager.cc:1918] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:28:54.027248 15684 log.cc:1196] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/5813377c3f5043c9a926e4b6e5c72669
I20260929 06:28:54.027446 15684 ts_tablet_manager.cc:1939] T 5813377c3f5043c9a926e4b6e5c72669 P 7c3ac9cbf8314d20863537e39646a80c: Deleting consensus metadata
I20260929 06:28:54.027951 15086 catalog_manager.cc:4841] TS 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647): tablet 5813377c3f5043c9a926e4b6e5c72669 (table test-table [id=30a089c06d4a4ec69f17884191a198f8]) successfully deleted
I20260929 06:28:54.028059 15697 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Bootstrap starting.
I20260929 06:28:54.028064 15693 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1: Bootstrap starting.
I20260929 06:28:54.028374 15697 tablet_bootstrap.cc:654] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:54.028469 15693 tablet_bootstrap.cc:654] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:54.028641 15694 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: Bootstrap starting.
I20260929 06:28:54.028959 15694 tablet_bootstrap.cc:654] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:54.028993 15697 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: No bootstrap required, opened a new log
I20260929 06:28:54.029031 15697 ts_tablet_manager.cc:1397] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:28:54.029146 15697 raft_consensus.cc:348] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.029215 15697 raft_consensus.cc:374] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:54.029238 15697 raft_consensus.cc:729] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d214e3f6374a30bd6721d5951a68bd, State: Initialized, Role: FOLLOWER
I20260929 06:28:54.029273 15697 consensus_queue.cc:260] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.029358 15697 ts_tablet_manager.cc:1428] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:54.029489 15694 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: No bootstrap required, opened a new log
I20260929 06:28:54.029529 15694 ts_tablet_manager.cc:1397] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:28:54.029620 15694 raft_consensus.cc:348] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.029675 15694 raft_consensus.cc:374] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:54.029693 15694 raft_consensus.cc:729] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3ac9cbf8314d20863537e39646a80c, State: Initialized, Role: FOLLOWER
I20260929 06:28:54.029754 15694 consensus_queue.cc:260] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.029846 15694 ts_tablet_manager.cc:1428] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:54.030241 15693 tablet_bootstrap.cc:492] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1: No bootstrap required, opened a new log
I20260929 06:28:54.030320 15693 ts_tablet_manager.cc:1397] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:28:54.030486 15693 raft_consensus.cc:348] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.030565 15693 raft_consensus.cc:374] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:54.030591 15693 raft_consensus.cc:729] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Initialized, Role: FOLLOWER
I20260929 06:28:54.030638 15693 consensus_queue.cc:260] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.030750 15693 ts_tablet_manager.cc:1428] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:28:54.192042 15705 raft_consensus.cc:482] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:28:54.192138 15705 raft_consensus.cc:504] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.192310 15705 leader_election.cc:288] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:54.192524 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf702c9db6a49d08746678fd6bb45d9" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd" is_pre_election: true
I20260929 06:28:54.192528 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf702c9db6a49d08746678fd6bb45d9" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c" is_pre_election: true
I20260929 06:28:54.192632 15535 raft_consensus.cc:2454] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 0.
I20260929 06:28:54.192631 15396 raft_consensus.cc:2454] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 0.
I20260929 06:28:54.192816 15190 leader_election.cc:302] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3ac9cbf8314d20863537e39646a80c, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:54.192929 15705 raft_consensus.cc:2781] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:28:54.192968 15705 raft_consensus.cc:482] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:28:54.192991 15705 raft_consensus.cc:3037] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:54.193742 15705 raft_consensus.cc:504] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.193894 15705 leader_election.cc:288] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 election: Requested vote from peers 7c3ac9cbf8314d20863537e39646a80c (127.13.15.130:34647), 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149)
I20260929 06:28:54.194039 15535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf702c9db6a49d08746678fd6bb45d9" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d214e3f6374a30bd6721d5951a68bd"
I20260929 06:28:54.194044 15396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf702c9db6a49d08746678fd6bb45d9" candidate_uuid: "93fa3708bf6b4a89878b78d478d49bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3ac9cbf8314d20863537e39646a80c"
I20260929 06:28:54.194129 15535 raft_consensus.cc:3037] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:54.194175 15396 raft_consensus.cc:3037] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:54.194732 15535 raft_consensus.cc:2454] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 1.
I20260929 06:28:54.194893 15396 raft_consensus.cc:2454] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93fa3708bf6b4a89878b78d478d49bb1 in term 1.
I20260929 06:28:54.195009 15187 leader_election.cc:302] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d214e3f6374a30bd6721d5951a68bd, 93fa3708bf6b4a89878b78d478d49bb1; no voters:
I20260929 06:28:54.195132 15705 raft_consensus.cc:2781] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:54.195254 15705 raft_consensus.cc:686] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 93fa3708bf6b4a89878b78d478d49bb1, State: Running, Role: LEADER
I20260929 06:28:54.195305 15705 consensus_queue.cc:237] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } }
I20260929 06:28:54.195942 15098 catalog_manager.cc:5495] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "93fa3708bf6b4a89878b78d478d49bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93fa3708bf6b4a89878b78d478d49bb1" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 } health_report { overall_health: UNKNOWN } } }
I20260929 06:28:54.211735 15535 raft_consensus.cc:1261] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Refusing update from remote peer 93fa3708bf6b4a89878b78d478d49bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:54.211755 15396 raft_consensus.cc:1261] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Refusing update from remote peer 93fa3708bf6b4a89878b78d478d49bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:28:54.211935 15705 consensus_queue.cc:1035] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3ac9cbf8314d20863537e39646a80c" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:28:54.212040 15696 consensus_queue.cc:1035] T 3bf702c9db6a49d08746678fd6bb45d9 P 93fa3708bf6b4a89878b78d478d49bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d214e3f6374a30bd6721d5951a68bd" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 06:28:54.216585 15666 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 06:28:55.733336 15098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:55.733444 15098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:28:55.734361 15098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e52a3e3d881a46628793ee1c809bcbf2: Sending DeleteTablet for 3 replicas of tablet 3bf702c9db6a49d08746678fd6bb45d9
I20260929 06:28:55.734608 15236 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:55 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:58518
I20260929 06:28:55.734637 15376 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:55 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:47636
I20260929 06:28:55.734784 15516 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf702c9db6a49d08746678fd6bb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:28:55 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57828
I20260929 06:28:55.734804 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15171
I20260929 06:28:55.735368 15748 tablet_replica.cc:330] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: stopping tablet replica
I20260929 06:28:55.735440 15748 raft_consensus.cc:2229] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:55.735486 15748 raft_consensus.cc:2258] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:55.735718 15748 ts_tablet_manager.cc:1905] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:28:55.737658 15748 ts_tablet_manager.cc:1918] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 06:28:55.737722 15748 log.cc:1196] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/3bf702c9db6a49d08746678fd6bb45d9
I20260929 06:28:55.737936 15748 ts_tablet_manager.cc:1939] T 3bf702c9db6a49d08746678fd6bb45d9 P 85d214e3f6374a30bd6721d5951a68bd: Deleting consensus metadata
I20260929 06:28:55.738399 15083 catalog_manager.cc:4841] TS 85d214e3f6374a30bd6721d5951a68bd (127.13.15.131:35149): tablet 3bf702c9db6a49d08746678fd6bb45d9 (table test-table [id=bd921a454fd7431b81621edb5c743483]) successfully deleted
I20260929 06:28:55.741732 15747 tablet_replica.cc:330] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: stopping tablet replica
I20260929 06:28:55.741813 15747 raft_consensus.cc:2229] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:28:55.741858 15747 raft_consensus.cc:2258] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:28:55.742065 15747 ts_tablet_manager.cc:1905] T 3bf702c9db6a49d08746678fd6bb45d9 P 7c3ac9cbf8314d20863537e39646a80c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 06:28:55.742738 15086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:35769 (error 108)
I20260929 06:28:55.742952 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15311
I20260929 06:28:55.743917 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:55.743105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35769 (local address 127.0.0.1:37386)
0929 06:28:55.743830 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:55.743887 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:35769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":55,"threads_started":1}
W20260929 06:28:55.744025 15086 catalog_manager.cc:4584] TS 93fa3708bf6b4a89878b78d478d49bb1 (127.13.15.129:35769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bf702c9db6a49d08746678fd6bb45d9: Network error: Client connection negotiation failed: client connection to 127.13.15.129:35769: connect: Connection refused (error 111)
I20260929 06:28:55.749636 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15451
I20260929 06:28:55.749667 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:55.749555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34647 (local address 127.0.0.1:55638)
0929 06:28:55.749607 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:55.749635 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.130:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 06:28:55.755616 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15067
2026-09-29T06:28:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5934 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:55 dist-test-slave-sm0p krb5kdc[15753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:55 dist-test-slave-sm0p krb5kdc[15753](info): set up 2 sockets
Sep 29 06:28:55 dist-test-slave-sm0p krb5kdc[15753](info): commencing operation
krb5kdc: starting...
W20260929 06:28:56.781754 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:56 dist-test-slave-sm0p krb5kdc[15753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:56.893064 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45445
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:56.970588 15766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:56.970790 15766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:56.970825 15766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:56.971894 15766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:56.971957 15766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:56.972000 15766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:56.972033 15766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:56.972056 15766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:56.973455 15766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:56.973727 15766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:56.973941 15766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:28:56.978703 15766 server_base.cc:1034] running on GCE node
W20260929 06:28:56.978675 15774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:56.978677 15771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:56.978670 15772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:56.979151 15766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:56.979429 15766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:56.980603 15766 hybrid_clock.cc:648] HybridClock initialized: now 1790663336980598 us; error 35 us; skew 500 ppm
Sep 29 06:28:56 dist-test-slave-sm0p krb5kdc[15753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663336, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:56.986809 15766 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:56.987270 15766 webserver.cc:460] Webserver started at http://127.13.15.190:45469/ using document root <none> and password file <none>
I20260929 06:28:56.987461 15766 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:56.987500 15766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:56.987583 15766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:56.988526 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "1616083e7c54428d810221a9051f7a84"
format_stamp: "Formatted at 2026-09-29 06:28:56 on dist-test-slave-sm0p"
server_key: "102bc298903826c3a31a0acdc37381c0"
server_key_iv: "36129fd294061dee5ca936dc13ffcb0e"
server_key_version: "encryptionkey@0"
I20260929 06:28:56.988813 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "1616083e7c54428d810221a9051f7a84"
format_stamp: "Formatted at 2026-09-29 06:28:56 on dist-test-slave-sm0p"
server_key: "102bc298903826c3a31a0acdc37381c0"
server_key_iv: "36129fd294061dee5ca936dc13ffcb0e"
server_key_version: "encryptionkey@0"
I20260929 06:28:56.989979 15766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:28:56.990693 15781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:56.990856 15766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:56.990936 15766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "1616083e7c54428d810221a9051f7a84"
format_stamp: "Formatted at 2026-09-29 06:28:56 on dist-test-slave-sm0p"
server_key: "102bc298903826c3a31a0acdc37381c0"
server_key_iv: "36129fd294061dee5ca936dc13ffcb0e"
server_key_version: "encryptionkey@0"
I20260929 06:28:56.990989 15766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:56.997869 15766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:56.999547 15766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:56.999648 15766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:57.003259 15766 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:45445
I20260929 06:28:57.003315 15833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:45445 every 8 connection(s)
I20260929 06:28:57.003623 15766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:57.004482 15834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:57.006634 15834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84: Bootstrap starting.
I20260929 06:28:57.006978 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15766
I20260929 06:28:57.007050 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:57.007247 13374 external_mini_cluster.cc:1428] Setting key 3a01e8b2ba120ce9893020e7e959abea
I20260929 06:28:57.007277 15834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:57.007539 15834 log.cc:824] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:57.008317 15834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84: No bootstrap required, opened a new log
I20260929 06:28:57.010666 15834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } }
I20260929 06:28:57.010825 15834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:57.010856 15834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1616083e7c54428d810221a9051f7a84, State: Initialized, Role: FOLLOWER
I20260929 06:28:57.010982 15834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } }
I20260929 06:28:57.011047 15834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:57.011073 15834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:57.011116 15834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:57.011650 15834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } }
I20260929 06:28:57.011780 15834 leader_election.cc:302] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1616083e7c54428d810221a9051f7a84; no voters:
I20260929 06:28:57.011937 15834 leader_election.cc:288] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:57.011981 15839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:57.012166 15839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [term 1 LEADER]: Becoming Leader. State: Replica: 1616083e7c54428d810221a9051f7a84, State: Running, Role: LEADER
I20260929 06:28:57.012231 15834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:57.012269 15839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } }
I20260929 06:28:57.012634 15839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1616083e7c54428d810221a9051f7a84. Latest consensus state: current_term: 1 leader_uuid: "1616083e7c54428d810221a9051f7a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } } }
I20260929 06:28:57.012872 15839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:57.012789 15840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1616083e7c54428d810221a9051f7a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1616083e7c54428d810221a9051f7a84" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45445 } } }
I20260929 06:28:57.013239 15840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:57.013392 15848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:57.013875 15848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:57.015676 15848 catalog_manager.cc:1347] Generated new cluster ID: d9632e004bd341dcb654aa867430f07b
I20260929 06:28:57.015722 15848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:28:57.017966 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.007710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.13.15.190:45445)
0929 06:28:57.007879 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:57.007889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:57.007920 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:28:57.008379 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:57.008388 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:57.008465 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:57.008577 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:57.009323 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.010231 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.011262 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.011645 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.011721 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.014744 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:57.014748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:57.014749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:57.014760 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:57.016501 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.016877 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.016881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.016881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.016974 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.017181 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.017182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.017182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.017394 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:57.017559 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:57.017646 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 06:28:57.017773 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260929 06:28:57.052173 15848 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:57.052728 15848 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:57.054998 15848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1616083e7c54428d810221a9051f7a84: Generated new TSK 0
I20260929 06:28:57.055181 15848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:28:57.086282 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:57.173122 15860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:57.173306 15860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:57.173328 15860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:57.174496 15860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:57.174535 15860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:57.174561 15860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:28:57.175940 15860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:57.176164 15860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:57.176414 15860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:57.181394 15868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.181463 15860 server_base.cc:1034] running on GCE node
W20260929 06:28:57.181447 15866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:57.181452 15865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.181838 15860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:57.182090 15860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:57.183261 15860 hybrid_clock.cc:648] HybridClock initialized: now 1790663337183252 us; error 27 us; skew 500 ppm
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:57.185670 15860 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:28:57.186110 15860 webserver.cc:460] Webserver started at http://127.13.15.129:43117/ using document root <none> and password file <none>
I20260929 06:28:57.186352 15860 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:57.186419 15860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:57.186554 15860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:57.187572 15860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "fa93d531a5074738b9f489bb8300d784"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "1be48c758cc5b1c9705c84832e22211f"
server_key_iv: "4f9a2935745510e5d7640e73fd0c7d96"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.187924 15860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "fa93d531a5074738b9f489bb8300d784"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "1be48c758cc5b1c9705c84832e22211f"
server_key_iv: "4f9a2935745510e5d7640e73fd0c7d96"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.189133 15860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:57.189785 15875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.189960 15860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:57.190050 15860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "fa93d531a5074738b9f489bb8300d784"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "1be48c758cc5b1c9705c84832e22211f"
server_key_iv: "4f9a2935745510e5d7640e73fd0c7d96"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.190132 15860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:57.201731 15860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:57.203421 15860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:57.203580 15860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:57.203781 15860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:57.204085 15860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:57.204136 15860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.204170 15860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:57.204198 15860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.209872 15860 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:35215
I20260929 06:28:57.209918 15988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:35215 every 8 connection(s)
I20260929 06:28:57.210211 15860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:28:57.211187 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15860
I20260929 06:28:57.211272 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:28:57.211432 13374 external_mini_cluster.cc:1428] Setting key 31cea65fa6ef9be35a76aea904080b35
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:28:57.219771 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.210905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53319 (local address 127.13.15.190:45445)
0929 06:28:57.210972 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:57.210974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:57.211173 (+ 199us) server_negotiation.cc:407] Connection header received
0929 06:28:57.211743 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:57.211744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:57.211755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:57.211809 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:57.212727 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.213333 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.214491 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.214839 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.214921 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.216895 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:57.216896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:57.216897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:57.216904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:57.218468 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.218841 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.218842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.218842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.218909 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.219143 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.219144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.219144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.219280 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:57.219351 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:57.219582 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 06:28:57.219673 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:28:57.219856 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.210896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45445 (local address 127.13.15.129:53319)
0929 06:28:57.211061 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:57.211092 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:57.211643 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:57.211873 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:57.211877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:57.212467 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:57.212636 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.212640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.213450 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.213452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.214400 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.214403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.214937 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.214938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.215029 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.215499 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:57.215506 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:57.216795 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:57.218545 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.218550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.218551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.218754 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.218979 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.218979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.218980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.219067 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.219345 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:57.219349 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:57.219453 (+ 104us) client_negotiation.cc:769] Sending connection context
0929 06:28:57.219531 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 06:28:57.219690 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260929 06:28:57.220314 15989 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45445
I20260929 06:28:57.220403 15989 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:57.220527 15989 heartbeater.cc:511] Master 127.13.15.190:45445 requested a full tablet report, sending...
I20260929 06:28:57.221026 15798 ts_manager.cc:194] Registered new tserver with Master: fa93d531a5074738b9f489bb8300d784 (127.13.15.129:35215)
I20260929 06:28:57.221679 15798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:28:57.236501 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:57.314917 15994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:57.315109 15994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:57.315130 15994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:57.316262 15994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:57.316303 15994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:57.316355 15994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:28:57.317762 15994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:57.318003 15994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:57.318245 15994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:57.322970 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.323098 15994 server_base.cc:1034] running on GCE node
W20260929 06:28:57.322988 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:57.323067 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.323439 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:57.323664 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:57.324807 15994 hybrid_clock.cc:648] HybridClock initialized: now 1790663337324789 us; error 34 us; skew 500 ppm
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:57.327135 15994 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:28:57.327562 15994 webserver.cc:460] Webserver started at http://127.13.15.130:34135/ using document root <none> and password file <none>
I20260929 06:28:57.327777 15994 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:57.327845 15994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:57.327997 15994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:57.328949 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "a418770c887a4d719d8f7796b4558ba4"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "50ae98e8d6aa62304ddfe8d6a7a9b06c"
server_key_iv: "3cd182eb6e13f1d980cb4a4d72c5f4c2"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.329293 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "a418770c887a4d719d8f7796b4558ba4"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "50ae98e8d6aa62304ddfe8d6a7a9b06c"
server_key_iv: "3cd182eb6e13f1d980cb4a4d72c5f4c2"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.330647 15994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:57.331311 16009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.331472 15994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:57.331566 15994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "a418770c887a4d719d8f7796b4558ba4"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "50ae98e8d6aa62304ddfe8d6a7a9b06c"
server_key_iv: "3cd182eb6e13f1d980cb4a4d72c5f4c2"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.331640 15994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:57.383414 15994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:57.385272 15994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:57.385435 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:57.385687 15994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:57.386008 15994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:57.386193 15994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.386303 15994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:57.386336 15994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.392683 15994 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:42193
I20260929 06:28:57.392717 16122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:42193 every 8 connection(s)
I20260929 06:28:57.393085 15994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:57.401404 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 15994
I20260929 06:28:57.401520 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:28:57.401819 13374 external_mini_cluster.cc:1428] Setting key 7a84b2c2fc80481a67f5c2fc8d839a46
I20260929 06:28:57.402361 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.393908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:49295 (local address 127.13.15.190:45445)
0929 06:28:57.393954 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:57.393956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:57.394209 (+ 253us) server_negotiation.cc:407] Connection header received
0929 06:28:57.394780 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:57.394781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:57.394794 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:57.394849 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:57.395800 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.396290 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.397253 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.397469 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.397541 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.399670 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:57.399671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:57.399671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:57.399679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:57.400846 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.401256 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.401257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.401257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.401315 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.401585 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.401586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.401587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.401759 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:57.401853 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:57.402198 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 06:28:57.402298 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:28:57.402534 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.393904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45445 (local address 127.13.15.130:49295)
0929 06:28:57.394079 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:57.394113 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:57.394676 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:57.394921 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:57.394925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:57.395503 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:57.395701 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.395706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.396380 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.396382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.397156 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.397158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.397546 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.397547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.397675 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.398215 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:57.398223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:57.399578 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:57.400949 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.400955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.400958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.401166 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.401379 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.401380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.401380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.401489 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.401838 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:57.401843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:57.401994 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 06:28:57.402124 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 06:28:57.402304 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260929 06:28:57.402976 16123 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45445
I20260929 06:28:57.403086 16123 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:57.403268 16123 heartbeater.cc:511] Master 127.13.15.190:45445 requested a full tablet report, sending...
I20260929 06:28:57.403656 15798 ts_manager.cc:194] Registered new tserver with Master: a418770c887a4d719d8f7796b4558ba4 (127.13.15.130:42193)
I20260929 06:28:57.404239 15798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:49295
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:28:57.424579 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:57.504282 16128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:57.504479 16128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:57.504503 16128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:57.505697 16128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:57.505738 16128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:57.505762 16128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:28:57.507180 16128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:57.507361 16128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:57.507529 16128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:57.512511 16134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.512538 16128 server_base.cc:1034] running on GCE node
W20260929 06:28:57.512660 16136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:57.512538 16133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:57.512954 16128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:57.513204 16128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:57.514369 16128 hybrid_clock.cc:648] HybridClock initialized: now 1790663337514357 us; error 25 us; skew 500 ppm
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:57.517082 16128 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:28:57.517517 16128 webserver.cc:460] Webserver started at http://127.13.15.131:36757/ using document root <none> and password file <none>
I20260929 06:28:57.517707 16128 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:57.517758 16128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:57.517855 16128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:57.518910 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "877c440a6d704483a613d523f61f51ca"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "a5c99616a48047bd7cf8e2483c521b46"
server_key_iv: "670bfad5f78a7c79ec2d5c5e5182f07d"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.519227 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "877c440a6d704483a613d523f61f51ca"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "a5c99616a48047bd7cf8e2483c521b46"
server_key_iv: "670bfad5f78a7c79ec2d5c5e5182f07d"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.520458 16128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:28:57.521170 16143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.521332 16128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:57.521426 16128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "877c440a6d704483a613d523f61f51ca"
format_stamp: "Formatted at 2026-09-29 06:28:57 on dist-test-slave-sm0p"
server_key: "a5c99616a48047bd7cf8e2483c521b46"
server_key_iv: "670bfad5f78a7c79ec2d5c5e5182f07d"
server_key_version: "encryptionkey@0"
I20260929 06:28:57.521502 16128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:57.546901 16128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:57.549108 16128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:57.549264 16128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:57.549477 16128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:28:57.549785 16128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:28:57.549836 16128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.549875 16128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:28:57.549902 16128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:57.555688 16128 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:42765
I20260929 06:28:57.555727 16256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:42765 every 8 connection(s)
I20260929 06:28:57.556020 16128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:28:57.558950 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16128
I20260929 06:28:57.559020 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:28:57.559209 13374 external_mini_cluster.cc:1428] Setting key 8fe3bc3c8eaa6d9756d2c8621678316c
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[15753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:57.565186 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.556621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:52149 (local address 127.13.15.190:45445)
0929 06:28:57.556674 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:57.556676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:57.556891 (+ 215us) server_negotiation.cc:407] Connection header received
0929 06:28:57.557834 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:57.557836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:57.557849 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:57.557900 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:57.558557 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.559151 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.560338 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.560553 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.560636 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.562702 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:57.562704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:57.562704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:57.562712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:57.563861 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.564282 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.564283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.564283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.564340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:57.564579 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:57.564580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:57.564581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:57.564697 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:57.564773 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:57.565023 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 06:28:57.565098 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 06:28:57.565377 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:57.556626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45445 (local address 127.13.15.131:52149)
0929 06:28:57.556775 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 06:28:57.556808 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:28:57.557711 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:28:57.557964 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:28:57.557968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:28:57.558273 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:28:57.558457 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.558462 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.559238 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.559239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.560237 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:28:57.560240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:28:57.560642 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:28:57.560644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:28:57.560768 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:57.561257 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:28:57.561265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:28:57.562609 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:28:57.563954 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.563960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.563962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.564172 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.564413 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:28:57.564414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:28:57.564414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:28:57.564513 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:28:57.564765 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:28:57.564770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:28:57.564898 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 06:28:57.565004 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 06:28:57.565147 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260929 06:28:57.565776 16257 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45445
I20260929 06:28:57.565877 16257 heartbeater.cc:464] Registering TS with master...
I20260929 06:28:57.566059 16257 heartbeater.cc:511] Master 127.13.15.190:45445 requested a full tablet report, sending...
I20260929 06:28:57.566431 15798 ts_manager.cc:194] Registered new tserver with Master: 877c440a6d704483a613d523f61f51ca (127.13.15.131:42765)
I20260929 06:28:57.566753 15798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:52149
I20260929 06:28:57.570775 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 06:28:57.581029 15838 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:28:57.580202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.13.15.190:45445)
0929 06:28:57.580263 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:57.580265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:57.580392 (+ 127us) server_negotiation.cc:407] Connection header received
0929 06:28:57.580442 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:57.580442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:57.580450 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:57.580501 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:57.580950 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55298: BlockingRecv error: recv got EOF from 127.0.0.1:55298 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:28:57.581923 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15860
I20260929 06:28:57.586550 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15994
I20260929 06:28:57.590672 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16128
I20260929 06:28:57.594878 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 15766
2026-09-29T06:28:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1837 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[16269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[16269](info): set up 2 sockets
Sep 29 06:28:57 dist-test-slave-sm0p krb5kdc[16269](info): commencing operation
krb5kdc: starting...
W20260929 06:28:58.597913 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:28:58.709115 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36393
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:36393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:28:58.786345 16282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:28:58.786586 16282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:28:58.786624 16282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:28:58.787779 16282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:28:58.787844 16282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:28:58.787868 16282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:28:58.787895 16282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:28:58.787918 16282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:28:58.789326 16282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:36393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:28:58.789593 16282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:28:58.789798 16282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:28:58.794417 16288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:58.794400 16287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:28:58.794395 16290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:28:58.794924 16282 server_base.cc:1034] running on GCE node
I20260929 06:28:58.795143 16282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:28:58.795368 16282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:28:58.796545 16282 hybrid_clock.cc:648] HybridClock initialized: now 1790663338796537 us; error 38 us; skew 500 ppm
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663338, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:28:58.799026 16282 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:28:58.799438 16282 webserver.cc:460] Webserver started at http://127.13.15.190:44947/ using document root <none> and password file <none>
I20260929 06:28:58.799659 16282 fs_manager.cc:359] Metadata directory not provided
I20260929 06:28:58.799747 16282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:28:58.799903 16282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:28:58.800956 16282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "527dcfc8b51d489e89ced10356344957"
format_stamp: "Formatted at 2026-09-29 06:28:58 on dist-test-slave-sm0p"
server_key: "7fd6cf6536888d486cb97ae009eb70a2"
server_key_iv: "3b9a6f09b36365188b65dbd05e867562"
server_key_version: "encryptionkey@0"
I20260929 06:28:58.801316 16282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "527dcfc8b51d489e89ced10356344957"
format_stamp: "Formatted at 2026-09-29 06:28:58 on dist-test-slave-sm0p"
server_key: "7fd6cf6536888d486cb97ae009eb70a2"
server_key_iv: "3b9a6f09b36365188b65dbd05e867562"
server_key_version: "encryptionkey@0"
I20260929 06:28:58.802790 16282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:28:58.803546 16297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:28:58.803710 16282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:28:58.803807 16282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "527dcfc8b51d489e89ced10356344957"
format_stamp: "Formatted at 2026-09-29 06:28:58 on dist-test-slave-sm0p"
server_key: "7fd6cf6536888d486cb97ae009eb70a2"
server_key_iv: "3b9a6f09b36365188b65dbd05e867562"
server_key_version: "encryptionkey@0"
I20260929 06:28:58.803901 16282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:28:58.837972 16282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:28:58.840253 16282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:28:58.840416 16282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:28:58.844372 16282 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:36393
I20260929 06:28:58.844422 16349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:36393 every 8 connection(s)
I20260929 06:28:58.844925 16282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:28:58.845594 16350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:28:58.847916 16350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957: Bootstrap starting.
I20260929 06:28:58.848534 16350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957: Neither blocks nor log segments found. Creating new log.
I20260929 06:28:58.848827 16350 log.cc:824] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957: Log is configured to *not* fsync() on all Append() calls
I20260929 06:28:58.849499 16350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957: No bootstrap required, opened a new log
I20260929 06:28:58.851118 16350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } }
I20260929 06:28:58.851254 16350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:28:58.851302 16350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527dcfc8b51d489e89ced10356344957, State: Initialized, Role: FOLLOWER
I20260929 06:28:58.851459 16350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } }
I20260929 06:28:58.851518 16350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:28:58.851559 16350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:28:58.851608 16350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:28:58.852459 16350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } }
I20260929 06:28:58.852568 16350 leader_election.cc:302] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 527dcfc8b51d489e89ced10356344957; no voters:
I20260929 06:28:58.852764 16350 leader_election.cc:288] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:28:58.852830 16353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:28:58.852998 16353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [term 1 LEADER]: Becoming Leader. State: Replica: 527dcfc8b51d489e89ced10356344957, State: Running, Role: LEADER
I20260929 06:28:58.853024 16350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:28:58.853124 16353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } }
I20260929 06:28:58.853458 16354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "527dcfc8b51d489e89ced10356344957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } } }
I20260929 06:28:58.853550 16354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:58.853461 16355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 527dcfc8b51d489e89ced10356344957. Latest consensus state: current_term: 1 leader_uuid: "527dcfc8b51d489e89ced10356344957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "527dcfc8b51d489e89ced10356344957" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36393 } } }
I20260929 06:28:58.853675 16355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957 [sys.catalog]: This master's current role is: LEADER
I20260929 06:28:58.853770 16359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:28:58.853813 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16282
I20260929 06:28:58.853943 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:28:58.854156 13374 external_mini_cluster.cc:1428] Setting key 55fce54f1ca2a762469350ca23c15a88
I20260929 06:28:58.854336 16359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:28:58.856092 16359 catalog_manager.cc:1347] Generated new cluster ID: 81408e0ec0a5469ab61ae6e7799135fe
I20260929 06:28:58.856144 16359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:28:58.864218 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:58.854812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.13.15.190:36393)
0929 06:28:58.854989 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:58.854999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:58.855240 (+ 241us) server_negotiation.cc:407] Connection header received
0929 06:28:58.855362 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:58.855369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:58.855426 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:58.855578 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:58.856385 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:58.857269 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:58.858029 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:58.858563 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:58.858677 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:58.860945 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:58.860949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:58.860950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:28:58.860961 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:58.862695 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:58.863072 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:58.863075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:58.863076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:58.863170 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:28:58.863381 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:28:58.863382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:28:58.863382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:28:58.863609 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:58.863772 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:58.863866 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 06:28:58.864020 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260929 06:28:58.865031 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:28:58.868184 16359 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:28:58.868862 16359 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:28:58.875001 16359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 527dcfc8b51d489e89ced10356344957: Generated new TSK 0
I20260929 06:28:58.875168 16359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:28:58.879081 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:28:58.875176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.13.15.190:36393)
0929 06:28:58.875218 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:28:58.875220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:28:58.875482 (+ 262us) server_negotiation.cc:407] Connection header received
0929 06:28:58.875559 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:28:58.875559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:28:58.875570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:28:58.875624 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:28:58.875983 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:58.876801 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:58.877557 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:28:58.877789 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:28:58.877878 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:28:58.878609 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:28:58.878611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:28:58.878611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:28:58.878794 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:28:58.878809 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:28:58.878815 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:28:58.878897 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 06:28:58.878971 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 06:28:58.878997 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:28:58.881510 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16282
2026-09-29T06:28:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1284 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16383](info): set up 2 sockets
Sep 29 06:28:58 dist-test-slave-sm0p krb5kdc[16383](info): commencing operation
krb5kdc: starting...
W20260929 06:28:59.893723 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:28:59 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:28:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:00.009490 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43805
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:43805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:00.087267 16396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:00.087471 16396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:00.087512 16396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:00.088637 16396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:00.088676 16396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:00.088690 16396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:00.088703 16396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:00.088754 16396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:00.090314 16396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:43805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:00.090576 16396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:00.090785 16396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:00.095433 16401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.095501 16396 server_base.cc:1034] running on GCE node
W20260929 06:29:00.095503 16402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.095433 16404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.095850 16396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:00.096134 16396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:00.097301 16396 hybrid_clock.cc:648] HybridClock initialized: now 1790663340097292 us; error 51 us; skew 500 ppm
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:00.099660 16396 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:00.100124 16396 webserver.cc:460] Webserver started at http://127.13.15.190:38399/ using document root <none> and password file <none>
I20260929 06:29:00.100361 16396 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:00.100425 16396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:00.100591 16396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:00.101547 16396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "d16afa2829184cb1a64cf1307d2d26c7"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "b8506e3444db98ee3668a88cb8124792"
server_key_iv: "8f35eac41f6c0a2d9dc0273802ae95ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.101890 16396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "d16afa2829184cb1a64cf1307d2d26c7"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "b8506e3444db98ee3668a88cb8124792"
server_key_iv: "8f35eac41f6c0a2d9dc0273802ae95ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.103199 16396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:00.103859 16411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.104027 16396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:00.104122 16396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "d16afa2829184cb1a64cf1307d2d26c7"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "b8506e3444db98ee3668a88cb8124792"
server_key_iv: "8f35eac41f6c0a2d9dc0273802ae95ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.104202 16396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:00.116696 16396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:00.118355 16396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:00.118511 16396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:00.122304 16396 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:43805
I20260929 06:29:00.122350 16463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:43805 every 8 connection(s)
I20260929 06:29:00.122700 16396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:00.123591 16464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:00.123842 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16396
I20260929 06:29:00.123955 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:00.124142 13374 external_mini_cluster.cc:1428] Setting key 927a441e6ef1b2c41c4282a692386db8
I20260929 06:29:00.126523 16464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: Bootstrap starting.
I20260929 06:29:00.127164 16464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:00.127442 16464 log.cc:824] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:00.128151 16464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: No bootstrap required, opened a new log
I20260929 06:29:00.130632 16464 raft_consensus.cc:348] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } }
I20260929 06:29:00.130761 16464 raft_consensus.cc:374] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:00.130782 16464 raft_consensus.cc:729] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16afa2829184cb1a64cf1307d2d26c7, State: Initialized, Role: FOLLOWER
I20260929 06:29:00.130913 16464 consensus_queue.cc:260] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } }
I20260929 06:29:00.130981 16464 raft_consensus.cc:388] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:00.131001 16464 raft_consensus.cc:482] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:00.131065 16464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:00.131668 16464 raft_consensus.cc:504] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } }
I20260929 06:29:00.131780 16464 leader_election.cc:302] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d16afa2829184cb1a64cf1307d2d26c7; no voters:
I20260929 06:29:00.131964 16464 leader_election.cc:288] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:00.132026 16469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:00.132184 16469 raft_consensus.cc:686] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [term 1 LEADER]: Becoming Leader. State: Replica: d16afa2829184cb1a64cf1307d2d26c7, State: Running, Role: LEADER
I20260929 06:29:00.132217 16464 sys_catalog.cc:564] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:00.132362 16469 consensus_queue.cc:237] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } }
I20260929 06:29:00.133502 16471 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d16afa2829184cb1a64cf1307d2d26c7. Latest consensus state: current_term: 1 leader_uuid: "d16afa2829184cb1a64cf1307d2d26c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } } }
I20260929 06:29:00.133582 16471 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:00.133643 16470 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d16afa2829184cb1a64cf1307d2d26c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16afa2829184cb1a64cf1307d2d26c7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43805 } } }
I20260929 06:29:00.133848 16470 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7 [sys.catalog]: This master's current role is: LEADER
W20260929 06:29:00.134117 16484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:29:00.134186 16484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:29:00.134443 16485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:00.134642 16485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:00.135516 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.124645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.13.15.190:43805)
0929 06:29:00.124834 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:00.124843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:00.124876 (+ 33us) server_negotiation.cc:407] Connection header received
0929 06:29:00.125368 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:00.125377 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:00.125438 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:00.125592 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:00.126165 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.127484 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.128507 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.128872 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.128954 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.131704 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:00.131708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:00.131708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:00.131719 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:00.133791 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.134262 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.134265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.134265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.134377 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.134609 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.134610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.134611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.134890 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:00.135028 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:00.135192 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 06:29:00.135322 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260929 06:29:00.136366 16485 catalog_manager.cc:1347] Generated new cluster ID: c25d89205c344711a86cecc04764223d
I20260929 06:29:00.136415 16485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:00.153993 16485 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:00.154539 16485 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:00.163172 16485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d16afa2829184cb1a64cf1307d2d26c7: Generated new TSK 0
I20260929 06:29:00.163360 16485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:29:00.188241 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:00.265730 16490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:00.265913 16490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:00.265935 16490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:00.267061 16490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:00.267102 16490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:00.267127 16490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:00.268471 16490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:00.268663 16490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:00.268817 16490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:00.273670 16498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.273646 16496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.273717 16490 server_base.cc:1034] running on GCE node
W20260929 06:29:00.273636 16495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.274063 16490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:00.274344 16490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:00.275506 16490 hybrid_clock.cc:648] HybridClock initialized: now 1790663340275506 us; error 37 us; skew 500 ppm
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:00.277846 16490 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:29:00.278301 16490 webserver.cc:460] Webserver started at http://127.13.15.129:32801/ using document root <none> and password file <none>
I20260929 06:29:00.278510 16490 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:00.278592 16490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:00.278769 16490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:00.279697 16490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "6673e7f303aa46eca51ebd03c858308d"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "ee061ff1342f7cff5f3e1eea16660971"
server_key_iv: "4f5432c1831a00f78891e212adf230ff"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.280063 16490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "6673e7f303aa46eca51ebd03c858308d"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "ee061ff1342f7cff5f3e1eea16660971"
server_key_iv: "4f5432c1831a00f78891e212adf230ff"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.281275 16490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:00.281941 16505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.282104 16490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.282209 16490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "6673e7f303aa46eca51ebd03c858308d"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "ee061ff1342f7cff5f3e1eea16660971"
server_key_iv: "4f5432c1831a00f78891e212adf230ff"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.282292 16490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:00.291620 16490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:00.293313 16490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:00.293462 16490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:00.293674 16490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:00.293977 16490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:00.294028 16490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.294066 16490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:00.294092 16490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.299675 16490 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:44503
I20260929 06:29:00.299714 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:44503 every 8 connection(s)
I20260929 06:29:00.299976 16490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:00.302269 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16490
I20260929 06:29:00.302367 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:00.302546 13374 external_mini_cluster.cc:1428] Setting key c42c35db1e0556d5751434c03c4c235b
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:00.309159 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.300579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:41461 (local address 127.13.15.190:43805)
0929 06:29:00.300621 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:00.300623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:00.300801 (+ 178us) server_negotiation.cc:407] Connection header received
0929 06:29:00.301291 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:00.301292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:00.301304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:00.301359 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:00.302237 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.302868 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.303829 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.304086 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.304158 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.306160 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:00.306162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:00.306163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:00.306171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:00.307745 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.308234 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.308235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.308235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.308289 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.308585 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.308586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.308586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.308677 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:00.308724 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:00.309021 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 06:29:00.309099 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:29:00.309408 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.300544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43805 (local address 127.13.15.129:41461)
0929 06:29:00.300682 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:00.300722 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:00.301187 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:00.301419 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:00.301422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:00.301939 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:00.302112 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.302117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.302970 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.302971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.303742 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.303743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.304179 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.304181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.304328 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.304754 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:00.304760 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:00.306039 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:00.307845 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.307851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.307854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.308153 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.308353 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.308354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.308354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.308519 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.308741 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:00.308748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:00.308902 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 06:29:00.309006 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 06:29:00.309178 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260929 06:29:00.309969 16619 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43805
I20260929 06:29:00.310065 16619 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:00.310256 16619 heartbeater.cc:511] Master 127.13.15.190:43805 requested a full tablet report, sending...
I20260929 06:29:00.310719 16428 ts_manager.cc:194] Registered new tserver with Master: 6673e7f303aa46eca51ebd03c858308d (127.13.15.129:44503)
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:29:00.311365 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:41461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:29:00.328645 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:00.409914 16624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:00.410141 16624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:00.410204 16624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:00.411375 16624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:00.411444 16624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:00.411492 16624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:00.413009 16624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:00.413262 16624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:00.413488 16624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:00.418445 16629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.418548 16632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.418490 16630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.418671 16624 server_base.cc:1034] running on GCE node
I20260929 06:29:00.418943 16624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:00.419193 16624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:00.420331 16624 hybrid_clock.cc:648] HybridClock initialized: now 1790663340420314 us; error 37 us; skew 500 ppm
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:00.422825 16624 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:29:00.423233 16624 webserver.cc:460] Webserver started at http://127.13.15.130:39809/ using document root <none> and password file <none>
I20260929 06:29:00.423449 16624 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:00.423494 16624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:00.423635 16624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:00.424613 16624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "5add4791a09a4a4999818cfae8f95818"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "f563fba0d3d6a2fce727d16955b977a8"
server_key_iv: "babcd70d3b696f26e8f23aa10732216d"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.424953 16624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "5add4791a09a4a4999818cfae8f95818"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "f563fba0d3d6a2fce727d16955b977a8"
server_key_iv: "babcd70d3b696f26e8f23aa10732216d"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.426246 16624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:29:00.426927 16639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.427096 16624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:00.427188 16624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "5add4791a09a4a4999818cfae8f95818"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "f563fba0d3d6a2fce727d16955b977a8"
server_key_iv: "babcd70d3b696f26e8f23aa10732216d"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.427271 16624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:00.444051 16624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:00.445775 16624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:00.445931 16624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:00.446208 16624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:00.446503 16624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:00.446557 16624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.446597 16624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:00.446625 16624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.452178 16624 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:45663
I20260929 06:29:00.452303 16752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:45663 every 8 connection(s)
I20260929 06:29:00.452543 16624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:00.453076 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16624
I20260929 06:29:00.453156 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:00.453370 13374 external_mini_cluster.cc:1428] Setting key df49d18af9fc88d6cd0dfb437f935d82
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
I20260929 06:29:00.462400 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.453267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:55773 (local address 127.13.15.190:43805)
0929 06:29:00.453319 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:00.453322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:00.453561 (+ 239us) server_negotiation.cc:407] Connection header received
0929 06:29:00.454704 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:00.454706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:00.454718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:00.454787 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:00.455463 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.455977 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.456997 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.457169 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.457220 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.459461 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:00.459462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:00.459462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:00.459470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:00.461001 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.461426 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.461427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.461427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.461483 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.461731 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.461732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.461732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.461848 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:00.461929 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:00.462234 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 06:29:00.462308 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:29:00.462587 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.453264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43805 (local address 127.13.15.130:55773)
0929 06:29:00.453437 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:00.453476 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:00.454606 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:00.454840 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:00.454844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:00.455076 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:00.455357 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.455363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.456067 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.456068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.456913 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.456915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.457243 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.457244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.457388 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.457846 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:00.457855 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:00.459364 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:00.461106 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.461112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.461115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.461338 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.461549 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.461549 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.461550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.461657 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.461932 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:00.461938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:00.462094 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 06:29:00.462215 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 06:29:00.462371 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260929 06:29:00.463047 16753 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43805
I20260929 06:29:00.463148 16753 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:00.463330 16753 heartbeater.cc:511] Master 127.13.15.190:43805 requested a full tablet report, sending...
I20260929 06:29:00.463688 16428 ts_manager.cc:194] Registered new tserver with Master: 5add4791a09a4a4999818cfae8f95818 (127.13.15.130:45663)
I20260929 06:29:00.464146 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:55773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:29:00.478037 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:00.557333 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:00.557503 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:00.557523 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:00.558950 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:00.558996 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:00.559034 16758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:00.561462 16758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:00.561698 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:00.561941 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:00.567905 16764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.567859 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:00.567844 16766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:00.568399 16758 server_base.cc:1034] running on GCE node
I20260929 06:29:00.568578 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:00.568746 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:00.569875 16758 hybrid_clock.cc:648] HybridClock initialized: now 1790663340569866 us; error 25 us; skew 500 ppm
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:00.572135 16758 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:29:00.572475 16758 webserver.cc:460] Webserver started at http://127.13.15.131:38323/ using document root <none> and password file <none>
I20260929 06:29:00.572631 16758 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:00.572669 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:00.572744 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:00.573626 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "65085e1911064db299c0a7130d7a133c"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "02876fe23d287cccf73a94e6abb8d997"
server_key_iv: "f60195ae56ab6925841538fa8788d5d9"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.573902 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "65085e1911064db299c0a7130d7a133c"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "02876fe23d287cccf73a94e6abb8d997"
server_key_iv: "f60195ae56ab6925841538fa8788d5d9"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.575163 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 06:29:00.575784 16773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.575924 16758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:00.576002 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "65085e1911064db299c0a7130d7a133c"
format_stamp: "Formatted at 2026-09-29 06:29:00 on dist-test-slave-sm0p"
server_key: "02876fe23d287cccf73a94e6abb8d997"
server_key_iv: "f60195ae56ab6925841538fa8788d5d9"
server_key_version: "encryptionkey@0"
I20260929 06:29:00.576051 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:00.589402 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:00.591089 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:00.591188 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:00.591341 16758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:00.591563 16758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:00.591591 16758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.591610 16758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:00.591622 16758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:00.596876 16758 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:46349
I20260929 06:29:00.596941 16886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:46349 every 8 connection(s)
I20260929 06:29:00.597234 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:00.602566 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16758
I20260929 06:29:00.602641 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:00.602798 13374 external_mini_cluster.cc:1428] Setting key 28ad45c8170256e6dd10becc8192f3bd
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:00.606060 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.597918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:49487 (local address 127.13.15.190:43805)
0929 06:29:00.597958 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:00.597960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:00.598187 (+ 227us) server_negotiation.cc:407] Connection header received
0929 06:29:00.598782 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:00.598783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:00.598794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:00.598847 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:00.599816 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.600328 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.601377 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.601586 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.601680 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.603850 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:00.603851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:00.603851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:00.603857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:00.605000 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.605334 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.605335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.605335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.605380 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.605591 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.605592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.605592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.605683 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:00.605732 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:00.605936 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 06:29:00.606001 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:29:00.606281 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.597888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43805 (local address 127.13.15.131:49487)
0929 06:29:00.598052 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:00.598083 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:00.598685 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:00.598914 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:00.598918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:00.599486 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:00.599720 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.599725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.600417 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.600419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.601291 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:00.601293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.601687 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.601688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:00.601795 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.602210 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:00.602217 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:00.603749 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:00.605069 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.605074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.605075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.605273 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.605439 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:00.605440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:00.605440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:00.605540 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:00.605735 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:00.605738 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:00.605836 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 06:29:00.605913 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 06:29:00.606020 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260929 06:29:00.606729 16887 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43805
I20260929 06:29:00.606834 16887 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:00.606998 16887 heartbeater.cc:511] Master 127.13.15.190:43805 requested a full tablet report, sending...
I20260929 06:29:00.607290 16428 ts_manager.cc:194] Registered new tserver with Master: 65085e1911064db299c0a7130d7a133c (127.13.15.131:46349)
I20260929 06:29:00.607597 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:49487
I20260929 06:29:00.614607 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663340, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:00.637418 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:00.629723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.13.15.190:43805)
0929 06:29:00.629855 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:00.629857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:00.629872 (+ 15us) server_negotiation.cc:407] Connection header received
0929 06:29:00.629956 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:00.629956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:00.629966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:00.630014 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:00.630610 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.631146 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.631827 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:00.632038 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:00.632087 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:00.634540 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:00.634542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:00.634542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:00.634549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:00.636083 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.636507 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.636508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.636509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.636573 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:00.636818 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:00.636820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:00.636820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:00.636974 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:00.637091 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:00.637209 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 06:29:00.637300 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260929 06:29:00.637676 16428 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54056
I20260929 06:29:00.638098 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16490
I20260929 06:29:00.643285 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16624
I20260929 06:29:00.647725 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16758
I20260929 06:29:00.651837 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16396
2026-09-29T06:29:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1771 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16899](info): set up 2 sockets
Sep 29 06:29:00 dist-test-slave-sm0p krb5kdc[16899](info): commencing operation
krb5kdc: starting...
W20260929 06:29:01.673967 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:01 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:01.784870 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:33131
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:01.864319 16912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:01.864549 16912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:01.864588 16912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:01.865803 16912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:01.865862 16912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:01.865886 16912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:01.865912 16912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:01.865943 16912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:01.867410 16912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:33131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:01.867681 16912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:01.867884 16912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:01.872594 16920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:01.872608 16912 server_base.cc:1034] running on GCE node
W20260929 06:29:01.872577 16918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:01.872577 16917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:01.873009 16912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:01.873322 16912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:01.874487 16912 hybrid_clock.cc:648] HybridClock initialized: now 1790663341874471 us; error 44 us; skew 500 ppm
Sep 29 06:29:01 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:01.876960 16912 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:01.877305 16912 webserver.cc:460] Webserver started at http://127.13.15.190:37009/ using document root <none> and password file <none>
I20260929 06:29:01.877462 16912 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:01.877501 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:01.877579 16912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:01.878563 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "26815f1cf1644fbba506e0c9f8f9da6b"
format_stamp: "Formatted at 2026-09-29 06:29:01 on dist-test-slave-sm0p"
server_key: "18f35daf416add3484f9ebaddbfbd70a"
server_key_iv: "34df37f84f376abd64193ac2ac06d790"
server_key_version: "encryptionkey@0"
I20260929 06:29:01.878836 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "26815f1cf1644fbba506e0c9f8f9da6b"
format_stamp: "Formatted at 2026-09-29 06:29:01 on dist-test-slave-sm0p"
server_key: "18f35daf416add3484f9ebaddbfbd70a"
server_key_iv: "34df37f84f376abd64193ac2ac06d790"
server_key_version: "encryptionkey@0"
I20260929 06:29:01.880023 16912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:01.880671 16927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:01.880853 16912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:01.880937 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "26815f1cf1644fbba506e0c9f8f9da6b"
format_stamp: "Formatted at 2026-09-29 06:29:01 on dist-test-slave-sm0p"
server_key: "18f35daf416add3484f9ebaddbfbd70a"
server_key_iv: "34df37f84f376abd64193ac2ac06d790"
server_key_version: "encryptionkey@0"
I20260929 06:29:01.881014 16912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:01.888944 16912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:01.890738 16912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:01.890887 16912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:01.894481 16912 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:33131
I20260929 06:29:01.894529 16979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:33131 every 8 connection(s)
I20260929 06:29:01.895004 16912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:01.895570 16980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:01.897605 16980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b: Bootstrap starting.
I20260929 06:29:01.898206 16980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:01.898492 16980 log.cc:824] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:01.898720 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 16912
I20260929 06:29:01.898844 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:01.899048 13374 external_mini_cluster.cc:1428] Setting key 32d977856b40f71eaed3c187f1d1fd20
I20260929 06:29:01.899269 16980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b: No bootstrap required, opened a new log
I20260929 06:29:01.901489 16980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } }
I20260929 06:29:01.901648 16980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:01.901684 16980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26815f1cf1644fbba506e0c9f8f9da6b, State: Initialized, Role: FOLLOWER
I20260929 06:29:01.901844 16980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } }
I20260929 06:29:01.901934 16980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:01.901965 16980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:01.902007 16980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:01.902714 16980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } }
I20260929 06:29:01.902798 16980 leader_election.cc:302] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26815f1cf1644fbba506e0c9f8f9da6b; no voters:
I20260929 06:29:01.902964 16980 leader_election.cc:288] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:01.903017 16985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:01.903190 16985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [term 1 LEADER]: Becoming Leader. State: Replica: 26815f1cf1644fbba506e0c9f8f9da6b, State: Running, Role: LEADER
I20260929 06:29:01.903226 16980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:01.903331 16985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } }
I20260929 06:29:01.903609 16985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26815f1cf1644fbba506e0c9f8f9da6b. Latest consensus state: current_term: 1 leader_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } } }
I20260929 06:29:01.903658 16985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:01.903622 16987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26815f1cf1644fbba506e0c9f8f9da6b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 33131 } } }
I20260929 06:29:01.903731 16987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:01.903849 16991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:01.904335 16991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:01.906453 16991 catalog_manager.cc:1347] Generated new cluster ID: 0e52f824077e43adb2a996b37f44501d
I20260929 06:29:01.906500 16991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:29:01 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:01.911809 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:01.899531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.13.15.190:33131)
0929 06:29:01.899716 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:01.899737 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:01.899771 (+ 34us) server_negotiation.cc:407] Connection header received
0929 06:29:01.901475 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:01.901481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:01.901530 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:01.901636 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:01.902480 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:01.903735 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:01.904831 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:01.905437 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:01.905589 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:01.908267 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:01.908271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:01.908271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:01.908283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:01.910188 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:01.910569 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:01.910573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:01.910573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:01.910708 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:01.910911 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:01.910912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:01.910913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:01.911218 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:01.911395 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:01.911463 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 06:29:01.911614 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260929 06:29:01.918290 16991 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:01.918970 16991 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:01.922587 16991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26815f1cf1644fbba506e0c9f8f9da6b: Generated new TSK 0
I20260929 06:29:01.922747 16991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:29:01.948671 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:02.026121 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:02.026360 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:02.026383 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:02.027520 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:02.027559 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:02.027586 17006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:02.029040 17006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:02.029269 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:02.029484 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:02.034247 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:02.034267 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:02.034354 17006 server_base.cc:1034] running on GCE node
W20260929 06:29:02.034287 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:02.034732 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:02.035001 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:02.036164 17006 hybrid_clock.cc:648] HybridClock initialized: now 1790663342036153 us; error 24 us; skew 500 ppm
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:02.045609 17006 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:29:02.046020 17006 webserver.cc:460] Webserver started at http://127.13.15.129:46163/ using document root <none> and password file <none>
I20260929 06:29:02.046294 17006 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:02.046360 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:02.046510 17006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:02.047521 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "ced227171d284c3ba72bcda369776c00"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "5e40dd6e1218893bc7de858f2da6c6b6"
server_key_iv: "7fc42ecf95f8730a96d62b80c342c2f5"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.047863 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "ced227171d284c3ba72bcda369776c00"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "5e40dd6e1218893bc7de858f2da6c6b6"
server_key_iv: "7fc42ecf95f8730a96d62b80c342c2f5"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.049120 17006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:02.049796 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.049965 17006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:02.050060 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "ced227171d284c3ba72bcda369776c00"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "5e40dd6e1218893bc7de858f2da6c6b6"
server_key_iv: "7fc42ecf95f8730a96d62b80c342c2f5"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.050175 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:02.065119 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:02.066924 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:02.067080 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:02.067286 17006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:02.067574 17006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:02.067626 17006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.067658 17006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:02.067685 17006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.073405 17006 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38823
I20260929 06:29:02.073449 17134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38823 every 8 connection(s)
I20260929 06:29:02.073743 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:02.082001 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.074400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:46305 (local address 127.13.15.190:33131)
0929 06:29:02.074492 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:02.074494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:02.074689 (+ 195us) server_negotiation.cc:407] Connection header received
0929 06:29:02.075514 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:02.075514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:02.075528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:02.075579 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:02.076115 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.076599 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.077509 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.077761 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.077814 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.079731 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:02.079732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:02.079733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:02.079739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:02.080920 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.081262 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.081263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.081263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.081310 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.081504 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.081504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.081505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.081603 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:02.081651 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:02.081867 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 06:29:02.081935 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:29:02.082123 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.074365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:33131 (local address 127.13.15.129:46305)
0929 06:29:02.074549 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:02.074588 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:02.075431 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:02.075637 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:02.075640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:02.075869 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:02.076045 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.076049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.076673 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.076674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.077436 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.077438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.077832 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.077833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.077933 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.078341 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:02.078349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:02.079648 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:02.080990 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.080995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.080997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.081200 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.081364 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.081365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.081365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.081452 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.081655 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:02.081659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:02.081763 (+ 104us) client_negotiation.cc:769] Sending connection context
0929 06:29:02.081839 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:29:02.081946 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260929 06:29:02.082592 17135 heartbeater.cc:347] Connected to a master server at 127.13.15.190:33131
I20260929 06:29:02.082676 17135 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:02.082801 17135 heartbeater.cc:511] Master 127.13.15.190:33131 requested a full tablet report, sending...
I20260929 06:29:02.082995 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17006
I20260929 06:29:02.083099 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:02.083294 13374 external_mini_cluster.cc:1428] Setting key 746af7443832a311edf4afa5078cec9c
I20260929 06:29:02.083359 16944 ts_manager.cc:194] Registered new tserver with Master: ced227171d284c3ba72bcda369776c00 (127.13.15.129:38823)
I20260929 06:29:02.084307 16944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:46305
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:29:02.106652 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:02.185038 17140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:02.185230 17140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:02.185251 17140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:02.186465 17140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:02.186503 17140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:02.186529 17140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:02.187986 17140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:02.188208 17140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:02.188416 17140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:02.193341 17146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:02.193323 17145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:02.193325 17148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:02.193543 17140 server_base.cc:1034] running on GCE node
I20260929 06:29:02.193756 17140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:02.193986 17140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:02.195147 17140 hybrid_clock.cc:648] HybridClock initialized: now 1790663342195123 us; error 40 us; skew 500 ppm
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:02.197547 17140 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:29:02.198001 17140 webserver.cc:460] Webserver started at http://127.13.15.130:44955/ using document root <none> and password file <none>
I20260929 06:29:02.198287 17140 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:02.198354 17140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:02.198493 17140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:02.199518 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "8f8e392fd6c14b1dbb76bdb72ed8c111"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "188e241aa8c92d3a1cf9cf0588ee2f1a"
server_key_iv: "8f33251266983222ae97a30899c6c826"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.199865 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "8f8e392fd6c14b1dbb76bdb72ed8c111"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "188e241aa8c92d3a1cf9cf0588ee2f1a"
server_key_iv: "8f33251266983222ae97a30899c6c826"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.201104 17140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:02.201793 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.201961 17140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:02.202049 17140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "8f8e392fd6c14b1dbb76bdb72ed8c111"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "188e241aa8c92d3a1cf9cf0588ee2f1a"
server_key_iv: "8f33251266983222ae97a30899c6c826"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.202132 17140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:02.210794 17140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:02.212435 17140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:02.212592 17140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:02.212800 17140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:02.213112 17140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:02.213167 17140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.213207 17140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:02.213233 17140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.219177 17140 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:37947
I20260929 06:29:02.219210 17268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:37947 every 8 connection(s)
I20260929 06:29:02.219537 17140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:02.220896 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17140
I20260929 06:29:02.221020 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:02.221266 13374 external_mini_cluster.cc:1428] Setting key 32a40e3082e3071036d3e52fa2c40530
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
I20260929 06:29:02.229466 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.220255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:50715 (local address 127.13.15.190:33131)
0929 06:29:02.220333 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:02.220335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:02.220491 (+ 156us) server_negotiation.cc:407] Connection header received
0929 06:29:02.221737 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:02.221738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:02.221750 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:02.221805 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:02.222711 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.223181 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.224270 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.224484 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.224571 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.226694 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:02.226695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:02.226695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:02.226703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:02.228077 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.228473 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.228475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.228475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.228555 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.228807 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.228808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.228809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.228925 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:02.229013 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:02.229339 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 06:29:02.229404 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:29:02.229638 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.220213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:33131 (local address 127.13.15.130:50715)
0929 06:29:02.220379 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:02.220412 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:02.221638 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:02.221882 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:02.221886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:02.222324 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:02.222609 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.222615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.223299 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.223300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.224178 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.224180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.224579 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.224580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.224698 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.225217 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:02.225226 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:02.226600 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:02.228168 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.228174 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.228175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.228387 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.228621 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.228622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.228623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.228742 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.229010 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:02.229018 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:02.229199 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 06:29:02.229326 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 06:29:02.229442 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260929 06:29:02.230188 17269 heartbeater.cc:347] Connected to a master server at 127.13.15.190:33131
I20260929 06:29:02.230279 17269 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:02.230417 17269 heartbeater.cc:511] Master 127.13.15.190:33131 requested a full tablet report, sending...
I20260929 06:29:02.230756 16944 ts_manager.cc:194] Registered new tserver with Master: 8f8e392fd6c14b1dbb76bdb72ed8c111 (127.13.15.130:37947)
I20260929 06:29:02.231135 16944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:50715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:29:02.244755 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:02.324852 17274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:02.325070 17274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:02.325109 17274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:02.326305 17274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:02.326368 17274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:02.326411 17274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:02.327877 17274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:02.328115 17274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:02.328349 17274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:02.333123 17279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:02.333123 17282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:02.333228 17280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:02.333271 17274 server_base.cc:1034] running on GCE node
I20260929 06:29:02.333575 17274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:02.333755 17274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:02.334883 17274 hybrid_clock.cc:648] HybridClock initialized: now 1790663342334869 us; error 29 us; skew 500 ppm
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:02.337234 17274 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:29:02.337625 17274 webserver.cc:460] Webserver started at http://127.13.15.131:32849/ using document root <none> and password file <none>
I20260929 06:29:02.337832 17274 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:02.337896 17274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:02.338050 17274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:02.339140 17274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "76810ec5635f40a08de7914e557ad30a"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "e1951db6322a6bcc70e510e2a24ccf74"
server_key_iv: "296066dac2e0752a7253829bbb3ad482"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.339484 17274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "76810ec5635f40a08de7914e557ad30a"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "e1951db6322a6bcc70e510e2a24ccf74"
server_key_iv: "296066dac2e0752a7253829bbb3ad482"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.340742 17274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:02.341398 17289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.341569 17274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:02.341665 17274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "76810ec5635f40a08de7914e557ad30a"
format_stamp: "Formatted at 2026-09-29 06:29:02 on dist-test-slave-sm0p"
server_key: "e1951db6322a6bcc70e510e2a24ccf74"
server_key_iv: "296066dac2e0752a7253829bbb3ad482"
server_key_version: "encryptionkey@0"
I20260929 06:29:02.341738 17274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:02.348906 17274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:02.351099 17274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:02.351248 17274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:02.351449 17274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:02.351723 17274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:02.351775 17274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.351814 17274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:02.351841 17274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:02.357456 17274 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:45975
I20260929 06:29:02.357510 17402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:45975 every 8 connection(s)
I20260929 06:29:02.357766 17274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:02.359278 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17274
I20260929 06:29:02.359390 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:02.359572 13374 external_mini_cluster.cc:1428] Setting key cbbf379c180041e65acf3ac88866e55e
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:02.366253 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.358403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:41763 (local address 127.13.15.190:33131)
0929 06:29:02.358497 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:02.358511 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:02.358669 (+ 158us) server_negotiation.cc:407] Connection header received
0929 06:29:02.359571 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:02.359572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:02.359585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:02.359646 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:02.360481 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.360962 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.361830 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.361999 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.362052 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.363982 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:02.363983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:02.363983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:02.363989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:02.365152 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.365477 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.365478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.365478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.365526 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.365726 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.365726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.365727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.365818 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:02.365866 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:02.366075 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 06:29:02.366171 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:29:02.366354 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.358385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:33131 (local address 127.13.15.131:41763)
0929 06:29:02.358559 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:02.358598 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:02.359460 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:02.359707 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:02.359710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:02.360061 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:02.360291 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.360312 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.361041 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.361041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.361761 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:02.361763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.362070 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.362071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:02.362217 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.362685 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:02.362694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:02.363909 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:02.365219 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.365224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.365225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.365416 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.365587 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:02.365588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:02.365588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:02.365674 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:02.365869 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:02.365872 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:02.365973 (+ 101us) client_negotiation.cc:769] Sending connection context
0929 06:29:02.366048 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 06:29:02.366177 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260929 06:29:02.366755 17403 heartbeater.cc:347] Connected to a master server at 127.13.15.190:33131
I20260929 06:29:02.366839 17403 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:02.367026 17403 heartbeater.cc:511] Master 127.13.15.190:33131 requested a full tablet report, sending...
I20260929 06:29:02.367372 16944 ts_manager.cc:194] Registered new tserver with Master: 76810ec5635f40a08de7914e557ad30a (127.13.15.131:45975)
I20260929 06:29:02.367740 16944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:41763
I20260929 06:29:02.371274 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:29:02.393013 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.385648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.13.15.129:38823)
0929 06:29:02.385834 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:02.385839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:02.385849 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:29:02.385874 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:02.385876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:02.385890 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:02.385954 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:02.386545 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.387257 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.387960 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.388258 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.388308 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.390500 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:02.390507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:02.390508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:02.390513 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:02.392059 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.392349 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.392352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.392352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.392403 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.392552 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.392553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.392553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.392665 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:02.392760 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:02.392827 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 06:29:02.392883 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[16899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:02.400697 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:02.394295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.13.15.190:33131)
0929 06:29:02.394357 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:02.394359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:02.394432 (+ 73us) server_negotiation.cc:407] Connection header received
0929 06:29:02.394509 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:02.394510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:02.394520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:02.394593 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:02.395052 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.395534 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.396202 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:02.396393 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:02.396474 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:02.398438 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:02.398439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:02.398439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:02.398447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:02.399554 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.399903 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.399905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.399905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.399958 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:02.400156 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:02.400157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:02.400158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:02.400297 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:02.400405 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:02.400514 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 06:29:02.400588 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260929 06:29:02.400892 16944 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60432
I20260929 06:29:02.401206 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17006
I20260929 06:29:02.405915 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17140
I20260929 06:29:02.410073 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17274
I20260929 06:29:02.414381 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 16912
2026-09-29T06:29:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1763 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[17415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[17415](info): set up 2 sockets
Sep 29 06:29:02 dist-test-slave-sm0p krb5kdc[17415](info): commencing operation
krb5kdc: starting...
W20260929 06:29:03.446053 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:03.558928 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:41977
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:41977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:03.637971 17428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:03.638211 17428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:03.638255 17428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:03.639364 17428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:03.639423 17428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:03.639448 17428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:03.639478 17428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:03.639500 17428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:03.640935 17428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:41977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:41977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:03.641191 17428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:03.641395 17428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:03.646056 17433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.646080 17436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.646126 17434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:03.646325 17428 server_base.cc:1034] running on GCE node
I20260929 06:29:03.646554 17428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:03.646778 17428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:03.647960 17428 hybrid_clock.cc:648] HybridClock initialized: now 1790663343647948 us; error 30 us; skew 500 ppm
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:03.650380 17428 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:03.650700 17428 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:03.650746 17428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:03.650902 17428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:03.651983 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "4d4bea48f0184d9589819100df3ee626"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "a13b00275f8db19d5b24e477406fb031"
server_key_iv: "35dd40a17b162fbd8fea300287f1f0da"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.652318 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "4d4bea48f0184d9589819100df3ee626"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "a13b00275f8db19d5b24e477406fb031"
server_key_iv: "35dd40a17b162fbd8fea300287f1f0da"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.653615 17428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:03.654364 17442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.654527 17428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.654621 17428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "4d4bea48f0184d9589819100df3ee626"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "a13b00275f8db19d5b24e477406fb031"
server_key_iv: "35dd40a17b162fbd8fea300287f1f0da"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.654691 17428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:03.670931 17428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:03.672632 17428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:03.672785 17428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:03.676721 17428 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:41977
I20260929 06:29:03.676743 17494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:41977 every 8 connection(s)
I20260929 06:29:03.677028 17428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:03.677914 17495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:03.680141 17495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: Bootstrap starting.
I20260929 06:29:03.680652 17495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:03.680884 17495 log.cc:824] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:03.681512 17495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: No bootstrap required, opened a new log
I20260929 06:29:03.683061 17495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } }
I20260929 06:29:03.683192 17495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:03.683213 17495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4bea48f0184d9589819100df3ee626, State: Initialized, Role: FOLLOWER
I20260929 06:29:03.683310 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17428
I20260929 06:29:03.683374 17495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } }
I20260929 06:29:03.683446 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:03.683457 17495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:03.683493 17495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:03.683565 17495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:03.683651 13374 external_mini_cluster.cc:1428] Setting key 8b112a0d75a79bb7710ece5d6a459a1b
I20260929 06:29:03.684511 17495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } }
I20260929 06:29:03.684659 17495 leader_election.cc:302] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d4bea48f0184d9589819100df3ee626; no voters:
I20260929 06:29:03.684823 17495 leader_election.cc:288] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:03.684890 17500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:03.685027 17500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [term 1 LEADER]: Becoming Leader. State: Replica: 4d4bea48f0184d9589819100df3ee626, State: Running, Role: LEADER
I20260929 06:29:03.685086 17495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:03.685125 17500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } }
I20260929 06:29:03.685397 17500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d4bea48f0184d9589819100df3ee626. Latest consensus state: current_term: 1 leader_uuid: "4d4bea48f0184d9589819100df3ee626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } } }
I20260929 06:29:03.685492 17500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:03.685407 17502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d4bea48f0184d9589819100df3ee626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4bea48f0184d9589819100df3ee626" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41977 } } }
I20260929 06:29:03.685647 17502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:03.685802 17506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:03.686285 17506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:03.688038 17506 catalog_manager.cc:1347] Generated new cluster ID: a8d2c079a529455382ade83b8eb1d306
I20260929 06:29:03.688091 17506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:03.694687 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:03.684105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.13.15.190:41977)
0929 06:29:03.684289 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:03.684296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:03.684319 (+ 23us) server_negotiation.cc:407] Connection header received
0929 06:29:03.684405 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:03.684410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:03.684448 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:03.684554 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:03.685364 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.686907 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.687836 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.688429 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.688544 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:03.691202 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:03.691205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:03.691206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:03.691217 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:03.693088 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:03.693494 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:03.693497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:03.693497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:03.693595 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:03.693797 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:03.693798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:03.693798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:03.694066 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:03.694252 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:03.694360 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 06:29:03.694481 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260929 06:29:03.695968 17506 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:03.696635 17506 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:03.707299 17506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: Generated new TSK 0
I20260929 06:29:03.707464 17506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:29:03.733815 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:03.815975 17521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:03.816192 17521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:03.816232 17521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:03.817302 17521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:03.817361 17521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:03.817404 17521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:03.818859 17521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:03.819100 17521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:03.819320 17521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:29:03.824288 17521 server_base.cc:1034] running on GCE node
W20260929 06:29:03.824280 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.824424 17526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.824448 17527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:03.824702 17521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:03.824945 17521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:03.826100 17521 hybrid_clock.cc:648] HybridClock initialized: now 1790663343826090 us; error 26 us; skew 500 ppm
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:03.828438 17521 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:29:03.828816 17521 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:03.828886 17521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:03.829017 17521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:03.830015 17521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "b28952fcaa1f46b58804599ba3760d56"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "7eeda84e73fbf87c00a1b7b224b3c8b7"
server_key_iv: "4180438ee1ad3e8b23c74a87256a3e2f"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.830400 17521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "b28952fcaa1f46b58804599ba3760d56"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "7eeda84e73fbf87c00a1b7b224b3c8b7"
server_key_iv: "4180438ee1ad3e8b23c74a87256a3e2f"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.831617 17521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:03.832296 17535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.832455 17521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.832559 17521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "b28952fcaa1f46b58804599ba3760d56"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "7eeda84e73fbf87c00a1b7b224b3c8b7"
server_key_iv: "4180438ee1ad3e8b23c74a87256a3e2f"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.832646 17521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:03.843287 17521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:03.844901 17521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:03.845062 17521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:03.845265 17521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:03.845551 17521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:03.845620 17521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.845661 17521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:03.845691 17521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.851323 17521 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:42871
I20260929 06:29:03.851365 17648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:42871 every 8 connection(s)
I20260929 06:29:03.851600 17521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:03.858477 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17521
I20260929 06:29:03.858600 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:03.858772 13374 external_mini_cluster.cc:1428] Setting key 54c7826459d1d2562a8b9d980e99e29d
I20260929 06:29:03.860457 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:03.852261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41977 (local address 127.13.15.129:53359)
0929 06:29:03.852413 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:03.852451 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:03.852866 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:03.853069 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:03.853073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:03.853300 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:03.853743 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:03.853748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.854440 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.854442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:03.855227 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:03.855228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.855552 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.855552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:03.855642 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:03.856039 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:03.856046 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:03.857351 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:03.859184 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:03.859190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:03.859192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:03.859460 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:03.859652 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:03.859652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:03.859652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:03.859737 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:03.860027 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:03.860035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:03.860131 (+ 96us) client_negotiation.cc:769] Sending connection context
0929 06:29:03.860204 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 06:29:03.860311 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260929 06:29:03.860471 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:03.852271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53359 (local address 127.13.15.190:41977)
0929 06:29:03.852329 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:03.852331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:03.852537 (+ 206us) server_negotiation.cc:407] Connection header received
0929 06:29:03.852940 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:03.852941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:03.852951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:03.853010 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:03.853813 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.854371 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.855295 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:03.855476 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:03.855534 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:03.857435 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:03.857436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:03.857437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:03.857444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:03.859089 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:03.859528 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:03.859530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:03.859530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:03.859591 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:03.859797 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:03.859798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:03.859798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:03.859931 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:03.860038 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:03.860248 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 06:29:03.860333 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:29:03.861284 17649 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41977
I20260929 06:29:03.861363 17649 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:03.861533 17649 heartbeater.cc:511] Master 127.13.15.190:41977 requested a full tablet report, sending...
I20260929 06:29:03.862066 17459 ts_manager.cc:194] Registered new tserver with Master: b28952fcaa1f46b58804599ba3760d56 (127.13.15.129:42871)
I20260929 06:29:03.862670 17459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53359
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:29:03.884625 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:03.962602 17654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:03.962813 17654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:03.962849 17654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:03.963958 17654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:03.964018 17654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:03.964071 17654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:03.965488 17654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:03.965772 17654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:03.965984 17654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:03.970680 17660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.970690 17662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:03.970670 17659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:03.970992 17654 server_base.cc:1034] running on GCE node
I20260929 06:29:03.971206 17654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:03.971438 17654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:03.972589 17654 hybrid_clock.cc:648] HybridClock initialized: now 1790663343972576 us; error 28 us; skew 500 ppm
Sep 29 06:29:03 dist-test-slave-sm0p krb5kdc[17415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:03.974828 17654 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:29:03.975188 17654 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:03.975257 17654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:03.975394 17654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:03.976341 17654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "20558fee695d4fa2b942870062050c3e"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "bf21caf8310ff1bb28e1e32bd7c7c64a"
server_key_iv: "53b0c38c19c9b880459d05b9fef72230"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.976689 17654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "20558fee695d4fa2b942870062050c3e"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "bf21caf8310ff1bb28e1e32bd7c7c64a"
server_key_iv: "53b0c38c19c9b880459d05b9fef72230"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.977912 17654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:03.978646 17668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.978819 17654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.978909 17654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "20558fee695d4fa2b942870062050c3e"
format_stamp: "Formatted at 2026-09-29 06:29:03 on dist-test-slave-sm0p"
server_key: "bf21caf8310ff1bb28e1e32bd7c7c64a"
server_key_iv: "53b0c38c19c9b880459d05b9fef72230"
server_key_version: "encryptionkey@0"
I20260929 06:29:03.978991 17654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:03.995673 17654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:03.997426 17654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:03.997589 17654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:03.997799 17654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:03.998085 17654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:03.998194 17654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:03.998252 17654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:03.998281 17654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.003734 17654 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:41449
I20260929 06:29:04.003782 17781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:41449 every 8 connection(s)
I20260929 06:29:04.004014 17654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:04.009307 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17654
I20260929 06:29:04.009407 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
Sep 29 06:29:04 dist-test-slave-sm0p krb5kdc[17415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663343, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:04.009614 13374 external_mini_cluster.cc:1428] Setting key 950be0d21b25db9102cbc901fdedec60
I20260929 06:29:04.012737 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.004677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41977 (local address 127.13.15.130:42245)
0929 06:29:04.004837 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.004876 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.005311 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.005500 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.005503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.005759 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.006241 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.006245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.006869 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.006870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.007660 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.007661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.007968 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.007969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.008060 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.008454 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:04.008462 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.009974 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.011623 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:04.011628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:04.011629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:04.011813 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:04.012009 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:04.012009 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:04.012010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:04.012105 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:04.012334 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.012337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.012428 (+ 91us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.012499 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.012601 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260929 06:29:04.012748 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.004729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:42245 (local address 127.13.15.190:41977)
0929 06:29:04.004771 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.004775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.004947 (+ 172us) server_negotiation.cc:407] Connection header received
0929 06:29:04.005391 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.005391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.005400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.005451 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.006316 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.006796 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.007730 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.007902 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.007951 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.010079 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.010080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.010080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:04.010086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.011544 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.011880 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.011881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.011882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.011942 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.012163 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.012164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.012164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.012281 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.012330 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.012536 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.012621 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:29:04.013178 17782 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41977
I20260929 06:29:04.013266 17782 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:04.013407 17782 heartbeater.cc:511] Master 127.13.15.190:41977 requested a full tablet report, sending...
I20260929 06:29:04.013724 17459 ts_manager.cc:194] Registered new tserver with Master: 20558fee695d4fa2b942870062050c3e (127.13.15.130:41449)
I20260929 06:29:04.014066 17459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:42245
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:29:04.033011 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:04.112239 17787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:04.112419 17787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:04.112442 17787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:04.113458 17787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:04.113495 17787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:04.113520 17787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:04.114955 17787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:04.115127 17787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:04.115285 17787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:04.119940 17792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.119916 17793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.119926 17795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.120165 17787 server_base.cc:1034] running on GCE node
I20260929 06:29:04.120364 17787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:04.120545 17787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:04.121706 17787 hybrid_clock.cc:648] HybridClock initialized: now 1790663344121688 us; error 28 us; skew 500 ppm
Sep 29 06:29:04 dist-test-slave-sm0p krb5kdc[17415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663344, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:04.124225 17787 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:29:04.124581 17787 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:04.124629 17787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:04.124718 17787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:04.125638 17787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "9e7696da6e4c45b9b7ee8128dec7cb18"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "027bb9c5a9d6c1d733b0e45c2b0e4ca3"
server_key_iv: "0ed01fda85d34b5a21a5ff690456129e"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.125908 17787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "9e7696da6e4c45b9b7ee8128dec7cb18"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "027bb9c5a9d6c1d733b0e45c2b0e4ca3"
server_key_iv: "0ed01fda85d34b5a21a5ff690456129e"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.127136 17787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:04.127761 17801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.127907 17787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:04.127997 17787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "9e7696da6e4c45b9b7ee8128dec7cb18"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "027bb9c5a9d6c1d733b0e45c2b0e4ca3"
server_key_iv: "0ed01fda85d34b5a21a5ff690456129e"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.128072 17787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:04.154716 17787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:04.156997 17787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:04.157111 17787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:04.157276 17787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:04.157594 17787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:04.157627 17787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.157680 17787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:04.157713 17787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.163496 17787 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:46639
I20260929 06:29:04.163581 17914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:46639 every 8 connection(s)
I20260929 06:29:04.163831 17787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:04.167505 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17787
I20260929 06:29:04.167634 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:04.167869 13374 external_mini_cluster.cc:1428] Setting key 285193ef83fcebfd199ace7601246689
Sep 29 06:29:04 dist-test-slave-sm0p krb5kdc[17415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663344, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:04.172824 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.164650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:54957 (local address 127.13.15.190:41977)
0929 06:29:04.164735 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.164737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.164982 (+ 245us) server_negotiation.cc:407] Connection header received
0929 06:29:04.165415 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.165416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.165428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.165514 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.166424 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.166991 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.167979 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.168253 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.168332 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.170367 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.170369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.170369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:04.170376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.171488 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.171900 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.171901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.171902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.171973 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.172214 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.172215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.172215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.172329 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.172400 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.172664 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.172741 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:29:04.172987 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.164644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41977 (local address 127.13.15.131:54957)
0929 06:29:04.164851 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.164885 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.165318 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.165551 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.165554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.165798 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.166335 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.166340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.167074 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.167074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.167895 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.167898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.168339 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.168340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.168447 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.168937 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:04.168944 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.170281 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.171589 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:04.171595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:04.171597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:04.171826 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:04.172036 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:04.172036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:04.172037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:04.172144 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:04.172391 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.172397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.172525 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.172628 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.172769 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":133,"threads_started":1}
I20260929 06:29:04.173386 17915 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41977
I20260929 06:29:04.173460 17915 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:04.173624 17915 heartbeater.cc:511] Master 127.13.15.190:41977 requested a full tablet report, sending...
I20260929 06:29:04.173943 17459 ts_manager.cc:194] Registered new tserver with Master: 9e7696da6e4c45b9b7ee8128dec7cb18 (127.13.15.131:46639)
I20260929 06:29:04.174309 17459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:54957
I20260929 06:29:04.179613 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:29:04.186811 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.180843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.13.15.190:41977)
0929 06:29:04.180902 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.180904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.181027 (+ 123us) server_negotiation.cc:407] Connection header received
0929 06:29:04.181067 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.181068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.181076 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.181122 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.181715 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.182215 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.182911 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.183091 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.183144 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.184289 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.184289 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.184290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:04.184294 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.185616 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.186006 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.186006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.186007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.186063 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:04.186289 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:04.186289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:04.186289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:04.186449 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.186523 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.186644 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.186731 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:29:04.188163 17459 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:29:04.189567 17459 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:04.195642 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:46639 (local address 127.0.0.1:40142)
0929 06:29:04.191678 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.191722 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.191787 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.192159 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.192162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.192164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:04.192228 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.192229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193203 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.193204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.194599 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.194601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195319 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.195320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.195404 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.195417 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.195507 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.195568 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
I20260929 06:29:04.195645 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.13.15.131:46639)
0929 06:29:04.191988 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.191992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.192001 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:04.192036 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.192039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.192055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.192134 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:04.192374 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193082 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.194683 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195219 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.195317 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.195377 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.195523 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.195563 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":75,"threads_started":1}
I20260929 06:29:04.196084 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:42871 (local address 127.0.0.1:42570)
0929 06:29:04.191824 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.191834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.191885 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.192444 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.192445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.192446 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:04.192474 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.192478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193723 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.193724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.194569 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.194572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195658 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.195659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.195888 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.195891 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.195922 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.195962 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":43,"threads_started":1}
I20260929 06:29:04.196234 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:41449 (local address 127.0.0.1:59420)
0929 06:29:04.192043 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.192052 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.192105 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.192328 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.192329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.192331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:04.192383 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.192385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193683 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.193684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.194760 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.194762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195831 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.195831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.195885 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.195889 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.195935 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.195962 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":36,"threads_started":1}
I20260929 06:29:04.196333 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42570 (local address 127.13.15.129:42871)
0929 06:29:04.192286 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.192290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.192298 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:04.192333 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.192334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.192348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.192565 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:04.192637 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193646 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.194845 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195556 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.196022 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.196089 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.196111 (+ 22us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.196242 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":84,"threads_started":1}
I20260929 06:29:04.196451 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.191869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.13.15.130:41449)
0929 06:29:04.192029 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.192033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.192094 (+ 61us) server_negotiation.cc:407] Connection header received
0929 06:29:04.192171 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.192173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.192186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.192267 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:04.192463 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.193583 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.194836 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.195560 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.195629 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.195682 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.196069 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.196115 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260929 06:29:04.196367 17849 tablet_service.cc:1467] Processing CreateTablet for tablet 6264eabe5d21476d9ab32b659a11270e (DEFAULT_TABLE table=test-table [id=34bbc5583f394952a3be4a23a116382b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.196729 17849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6264eabe5d21476d9ab32b659a11270e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.197088 17716 tablet_service.cc:1467] Processing CreateTablet for tablet 6264eabe5d21476d9ab32b659a11270e (DEFAULT_TABLE table=test-table [id=34bbc5583f394952a3be4a23a116382b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.197099 17583 tablet_service.cc:1467] Processing CreateTablet for tablet 6264eabe5d21476d9ab32b659a11270e (DEFAULT_TABLE table=test-table [id=34bbc5583f394952a3be4a23a116382b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.197392 17716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6264eabe5d21476d9ab32b659a11270e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.197392 17583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6264eabe5d21476d9ab32b659a11270e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.199771 17929 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Bootstrap starting.
I20260929 06:29:04.200321 17929 tablet_bootstrap.cc:654] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.200570 17930 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: Bootstrap starting.
I20260929 06:29:04.200892 17931 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Bootstrap starting.
I20260929 06:29:04.200946 17929 log.cc:824] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.201344 17930 tablet_bootstrap.cc:654] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.201499 17931 tablet_bootstrap.cc:654] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.201732 17929 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: No bootstrap required, opened a new log
I20260929 06:29:04.201809 17929 ts_tablet_manager.cc:1397] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:04.201901 17930 log.cc:824] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.202085 17931 log.cc:824] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.202742 17930 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: No bootstrap required, opened a new log
I20260929 06:29:04.202811 17930 ts_tablet_manager.cc:1397] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:29:04.202908 17931 tablet_bootstrap.cc:492] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: No bootstrap required, opened a new log
I20260929 06:29:04.202981 17931 ts_tablet_manager.cc:1397] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:29:04.203856 17929 raft_consensus.cc:348] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.204020 17929 raft_consensus.cc:374] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.204056 17929 raft_consensus.cc:729] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7696da6e4c45b9b7ee8128dec7cb18, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.204170 17929 consensus_queue.cc:260] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.204434 17929 ts_tablet_manager.cc:1428] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 06:29:04.204584 17915 heartbeater.cc:503] Master 127.13.15.190:41977 was elected leader, sending a full tablet report...
I20260929 06:29:04.204463 17931 raft_consensus.cc:348] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.204635 17931 raft_consensus.cc:374] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.204667 17931 raft_consensus.cc:729] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20558fee695d4fa2b942870062050c3e, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.204782 17931 consensus_queue.cc:260] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.204770 17930 raft_consensus.cc:348] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.204877 17930 raft_consensus.cc:374] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.204897 17930 raft_consensus.cc:729] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b28952fcaa1f46b58804599ba3760d56, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.205050 17931 ts_tablet_manager.cc:1428] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:29:04.205014 17930 consensus_queue.cc:260] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.205133 17782 heartbeater.cc:503] Master 127.13.15.190:41977 was elected leader, sending a full tablet report...
I20260929 06:29:04.205294 17649 heartbeater.cc:503] Master 127.13.15.190:41977 was elected leader, sending a full tablet report...
I20260929 06:29:04.205298 17930 ts_tablet_manager.cc:1428] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260929 06:29:04.254535 17783 tablet.cc:2369] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:04.254985 17937 raft_consensus.cc:482] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:04.255086 17937 raft_consensus.cc:504] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.255416 17937 leader_election.cc:288] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e7696da6e4c45b9b7ee8128dec7cb18 (127.13.15.131:46639), 20558fee695d4fa2b942870062050c3e (127.13.15.130:41449)
I20260929 06:29:04.258529 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.255721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:41449 (local address 127.13.15.129:60051)
0929 06:29:04.255931 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.255947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.256028 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.256179 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.256179 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.256181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:04.256243 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.256245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.256927 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.256928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.257702 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.257704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.258316 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.258317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.258385 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.258388 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.258439 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.258471 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":124,"threads_started":1}
I20260929 06:29:04.258610 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.255801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:60051 (local address 127.13.15.130:41449)
0929 06:29:04.255858 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.255860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.256051 (+ 191us) server_negotiation.cc:407] Connection header received
0929 06:29:04.256078 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.256079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.256096 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.256156 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:04.256308 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.256858 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.257797 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.258231 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.258288 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.258295 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.258479 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.258539 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:29:04.258714 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.255647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:40519 (local address 127.13.15.131:46639)
0929 06:29:04.255699 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.255701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.255714 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:29:04.255769 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.255770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.255779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.255848 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:04.256012 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.256632 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.257794 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.258260 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.258356 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.258364 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.258589 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.258629 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 06:29:04.258679 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.255589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:46639 (local address 127.13.15.129:40519)
0929 06:29:04.255651 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.255664 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.255706 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.255861 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.255862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.255864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:04.255935 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.255936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.256707 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.256708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.257697 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:04.257700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.258383 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.258384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:04.258507 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.258510 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.258575 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.258606 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":31}
I20260929 06:29:04.259254 17869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6264eabe5d21476d9ab32b659a11270e" candidate_uuid: "b28952fcaa1f46b58804599ba3760d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" is_pre_election: true
I20260929 06:29:04.259254 17736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6264eabe5d21476d9ab32b659a11270e" candidate_uuid: "b28952fcaa1f46b58804599ba3760d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20558fee695d4fa2b942870062050c3e" is_pre_election: true
I20260929 06:29:04.259400 17869 raft_consensus.cc:2454] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b28952fcaa1f46b58804599ba3760d56 in term 0.
I20260929 06:29:04.259400 17736 raft_consensus.cc:2454] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b28952fcaa1f46b58804599ba3760d56 in term 0.
I20260929 06:29:04.259605 17537 leader_election.cc:302] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e7696da6e4c45b9b7ee8128dec7cb18, b28952fcaa1f46b58804599ba3760d56; no voters:
I20260929 06:29:04.259742 17937 raft_consensus.cc:2781] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:04.259827 17937 raft_consensus.cc:482] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:04.259858 17937 raft_consensus.cc:3037] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:04.260507 17937 raft_consensus.cc:504] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.260634 17937 leader_election.cc:288] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [CANDIDATE]: Term 1 election: Requested vote from peers 9e7696da6e4c45b9b7ee8128dec7cb18 (127.13.15.131:46639), 20558fee695d4fa2b942870062050c3e (127.13.15.130:41449)
I20260929 06:29:04.260813 17869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6264eabe5d21476d9ab32b659a11270e" candidate_uuid: "b28952fcaa1f46b58804599ba3760d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18"
I20260929 06:29:04.260814 17736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6264eabe5d21476d9ab32b659a11270e" candidate_uuid: "b28952fcaa1f46b58804599ba3760d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20558fee695d4fa2b942870062050c3e"
I20260929 06:29:04.260910 17869 raft_consensus.cc:3037] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:04.260906 17736 raft_consensus.cc:3037] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:04.261576 17736 raft_consensus.cc:2454] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b28952fcaa1f46b58804599ba3760d56 in term 1.
I20260929 06:29:04.261641 17869 raft_consensus.cc:2454] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b28952fcaa1f46b58804599ba3760d56 in term 1.
I20260929 06:29:04.261758 17537 leader_election.cc:302] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20558fee695d4fa2b942870062050c3e, b28952fcaa1f46b58804599ba3760d56; no voters:
I20260929 06:29:04.261883 17937 raft_consensus.cc:2781] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:04.262058 17937 raft_consensus.cc:686] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [term 1 LEADER]: Becoming Leader. State: Replica: b28952fcaa1f46b58804599ba3760d56, State: Running, Role: LEADER
I20260929 06:29:04.262234 17937 consensus_queue.cc:237] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } }
I20260929 06:29:04.262972 17458 catalog_manager.cc:5495] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 reported cstate change: term changed from 0 to 1, leader changed from <none> to b28952fcaa1f46b58804599ba3760d56 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "b28952fcaa1f46b58804599ba3760d56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b28952fcaa1f46b58804599ba3760d56" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42871 } health_report { overall_health: HEALTHY } } }
I20260929 06:29:04.286485 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.284078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.13.15.129:42871)
0929 06:29:04.284119 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.284121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.284129 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:04.284192 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.284193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.284208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.284267 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:04.284358 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.284847 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.285651 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:04.285824 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:04.285875 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:04.286007 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:04.286114 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:04.286379 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.286401 (+ 22us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.286418 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:29:04.288125 17736 raft_consensus.cc:1261] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 1 FOLLOWER]: Refusing update from remote peer b28952fcaa1f46b58804599ba3760d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:04.288157 17869 raft_consensus.cc:1261] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 1 FOLLOWER]: Refusing update from remote peer b28952fcaa1f46b58804599ba3760d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:04.288491 17937 consensus_queue.cc:1035] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e7696da6e4c45b9b7ee8128dec7cb18" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 46639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:04.288648 17939 consensus_queue.cc:1035] T 6264eabe5d21476d9ab32b659a11270e P b28952fcaa1f46b58804599ba3760d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20558fee695d4fa2b942870062050c3e" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:04.290711 17942 mvcc.cc:205] Tried to move back new op lower bound from 7334557058201784320 to 7334557058098614272. Current Snapshot: MvccSnapshot[applied={T|T < 7334557058201784320}]
I20260929 06:29:04.291213 17944 mvcc.cc:205] Tried to move back new op lower bound from 7334557058201784320 to 7334557058098614272. Current Snapshot: MvccSnapshot[applied={T|T < 7334557058201784320}]
I20260929 06:29:04.293310 17458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:04.293385 17458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:04.294050 17458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d4bea48f0184d9589819100df3ee626: Sending DeleteTablet for 3 replicas of tablet 6264eabe5d21476d9ab32b659a11270e
I20260929 06:29:04.294374 17583 tablet_service.cc:1514] Processing DeleteTablet for tablet 6264eabe5d21476d9ab32b659a11270e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:42570
I20260929 06:29:04.294458 17849 tablet_service.cc:1514] Processing DeleteTablet for tablet 6264eabe5d21476d9ab32b659a11270e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:40142
I20260929 06:29:04.294577 17716 tablet_service.cc:1514] Processing DeleteTablet for tablet 6264eabe5d21476d9ab32b659a11270e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:59420
I20260929 06:29:04.294641 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17521
I20260929 06:29:04.295146 17954 tablet_replica.cc:330] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: stopping tablet replica
I20260929 06:29:04.295246 17954 raft_consensus.cc:2229] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:04.295347 17954 raft_consensus.cc:2258] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:04.295692 17954 ts_tablet_manager.cc:1905] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:04.297242 17954 ts_tablet_manager.cc:1918] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:04.297319 17954 log.cc:1196] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/6264eabe5d21476d9ab32b659a11270e
I20260929 06:29:04.297551 17954 ts_tablet_manager.cc:1939] T 6264eabe5d21476d9ab32b659a11270e P 20558fee695d4fa2b942870062050c3e: Deleting consensus metadata
I20260929 06:29:04.297962 17444 catalog_manager.cc:4841] TS 20558fee695d4fa2b942870062050c3e (127.13.15.130:41449): tablet 6264eabe5d21476d9ab32b659a11270e (table test-table [id=34bbc5583f394952a3be4a23a116382b]) successfully deleted
W20260929 06:29:04.300755 17446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:42871 (error 108)
I20260929 06:29:04.300941 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17654
I20260929 06:29:04.301028 17953 tablet_replica.cc:330] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: stopping tablet replica
I20260929 06:29:04.301116 17953 raft_consensus.cc:2229] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:04.301204 17953 raft_consensus.cc:2258] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:04.301532 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.301078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:42871 (local address 127.0.0.1:42578)
0929 06:29:04.301459 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.301495 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364}
W20260929 06:29:04.301865 17446 catalog_manager.cc:4584] TS b28952fcaa1f46b58804599ba3760d56 (127.13.15.129:42871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6264eabe5d21476d9ab32b659a11270e: Network error: Client connection negotiation failed: client connection to 127.13.15.129:42871: connect: Connection refused (error 111)
I20260929 06:29:04.302923 17953 ts_tablet_manager.cc:1905] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:04.304183 17953 ts_tablet_manager.cc:1918] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:04.304262 17953 log.cc:1196] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/6264eabe5d21476d9ab32b659a11270e
I20260929 06:29:04.304518 17953 ts_tablet_manager.cc:1939] T 6264eabe5d21476d9ab32b659a11270e P 9e7696da6e4c45b9b7ee8128dec7cb18: Deleting consensus metadata
I20260929 06:29:04.304888 17444 catalog_manager.cc:4841] TS 9e7696da6e4c45b9b7ee8128dec7cb18 (127.13.15.131:46639): tablet 6264eabe5d21476d9ab32b659a11270e (table test-table [id=34bbc5583f394952a3be4a23a116382b]) successfully deleted
I20260929 06:29:04.305774 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17787
I20260929 06:29:04.311081 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17428
2026-09-29T06:29:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1895 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T06:29:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:04Z Disabled control of system clock
I20260929 06:29:04.337246 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37605
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 06:29:04.414916 17959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:04.415098 17959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:04.415119 17959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:04.416157 17959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:04.416189 17959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:04.416203 17959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:04.416215 17959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:04.416226 17959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:04.417451 17959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:37605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:04.417634 17959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:04.417853 17959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:04.422408 17964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.422559 17959 server_base.cc:1034] running on GCE node
W20260929 06:29:04.422408 17965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.422505 17967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.422868 17959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:04.423132 17959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:04.424283 17959 hybrid_clock.cc:648] HybridClock initialized: now 1790663344424271 us; error 27 us; skew 500 ppm
I20260929 06:29:04.425455 17959 webserver.cc:460] Webserver started at http://127.13.15.190:43643/ using document root <none> and password file <none>
I20260929 06:29:04.425659 17959 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:04.425700 17959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:04.425783 17959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:04.426813 17959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "d59c8d9ec8a749a6bb3f1983e31e4568"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "7ab1d75d8dadddec92f7a279865b1419"
server_key_iv: "fc854187d507517f3ab0a05b496e2566"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.427095 17959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "d59c8d9ec8a749a6bb3f1983e31e4568"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "7ab1d75d8dadddec92f7a279865b1419"
server_key_iv: "fc854187d507517f3ab0a05b496e2566"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.428217 17959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:04.428804 17973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.428947 17959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:04.429028 17959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "d59c8d9ec8a749a6bb3f1983e31e4568"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "7ab1d75d8dadddec92f7a279865b1419"
server_key_iv: "fc854187d507517f3ab0a05b496e2566"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.429086 17959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:04.432232 17959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:04.432405 17959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:04.432494 17959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:04.435847 17959 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:37605
I20260929 06:29:04.435905 18025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:37605 every 8 connection(s)
I20260929 06:29:04.436223 17959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:04.436842 18026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.439404 18026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: Bootstrap starting.
I20260929 06:29:04.439942 18026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.440204 18026 log.cc:824] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.440824 18026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: No bootstrap required, opened a new log
I20260929 06:29:04.441303 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 17959
I20260929 06:29:04.441411 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:04.441579 13374 external_mini_cluster.cc:1428] Setting key 509bfd77a787f7c6b8dd8853ac713e33
I20260929 06:29:04.442832 18026 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } }
I20260929 06:29:04.442983 18026 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.443022 18026 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59c8d9ec8a749a6bb3f1983e31e4568, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.443163 18026 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } }
I20260929 06:29:04.443269 18026 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:04.443308 18026 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:04.443349 18026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:04.444031 18026 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } }
I20260929 06:29:04.444069 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.442053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.13.15.190:37605)
0929 06:29:04.442298 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.442305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.442326 (+ 21us) server_negotiation.cc:407] Connection header received
0929 06:29:04.442814 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.442820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.442852 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.442962 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.443620 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.443623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.443623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.443627 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.443668 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.443678 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.443733 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.443818 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.443884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":118,"threads_started":1}
I20260929 06:29:04.444156 18026 leader_election.cc:302] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59c8d9ec8a749a6bb3f1983e31e4568; no voters:
I20260929 06:29:04.444344 18026 leader_election.cc:288] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:04.444612 18026 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:04.444644 18031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:04.444801 18031 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [term 1 LEADER]: Becoming Leader. State: Replica: d59c8d9ec8a749a6bb3f1983e31e4568, State: Running, Role: LEADER
I20260929 06:29:04.444911 18031 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } }
I20260929 06:29:04.445289 18032 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } } }
I20260929 06:29:04.445422 18033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59c8d9ec8a749a6bb3f1983e31e4568. Latest consensus state: current_term: 1 leader_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c8d9ec8a749a6bb3f1983e31e4568" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37605 } } }
I20260929 06:29:04.445492 18032 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:04.445516 18033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:04.445849 18042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:04.446349 18042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:04.447722 18042 catalog_manager.cc:1347] Generated new cluster ID: cb6480a63cdf4ee890eec660d8c8cccb
I20260929 06:29:04.447773 18042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:04.465261 18042 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:04.465435 18042 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:04.471333 18042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: Generated new TSK 0
I20260929 06:29:04.471524 18042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:29:04.477188 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--builtin_ntp_servers=127.13.15.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 06:29:04.558037 18050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:04.558338 18050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:04.558379 18050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:04.559508 18050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:04.559568 18050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:04.559607 18050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:04.560959 18050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:04.561214 18050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:04.561478 18050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:29:04.567246 18050 server_base.cc:1034] running on GCE node
W20260929 06:29:04.567219 18056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.567210 18055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.567337 18058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.567847 18050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:04.568097 18050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:04.569288 18050 hybrid_clock.cc:648] HybridClock initialized: now 1790663344569264 us; error 37 us; skew 500 ppm
I20260929 06:29:04.570545 18050 webserver.cc:460] Webserver started at http://127.13.15.129:35797/ using document root <none> and password file <none>
I20260929 06:29:04.570767 18050 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:04.570833 18050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:04.570959 18050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:04.571969 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "cad27348e45e43a796ed891855e1debd"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "1c2af9b72ba51de16eb8914175f6d450"
server_key_iv: "7136887987615033c7d234aced23fecb"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.572312 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "cad27348e45e43a796ed891855e1debd"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "1c2af9b72ba51de16eb8914175f6d450"
server_key_iv: "7136887987615033c7d234aced23fecb"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.573560 18050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:04.574360 18064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.574536 18050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:04.574627 18050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "cad27348e45e43a796ed891855e1debd"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "1c2af9b72ba51de16eb8914175f6d450"
server_key_iv: "7136887987615033c7d234aced23fecb"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.574707 18050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:04.577836 18050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:04.578106 18050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:04.578269 18050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:04.578495 18050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:04.578830 18050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:04.578898 18050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.578936 18050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:04.578963 18050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.584451 18050 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:43659
I20260929 06:29:04.584515 18177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:43659 every 8 connection(s)
I20260929 06:29:04.584810 18050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:04.587570 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.585527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:44435 (local address 127.13.15.190:37605)
0929 06:29:04.585593 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.585595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.585831 (+ 236us) server_negotiation.cc:407] Connection header received
0929 06:29:04.586302 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.586303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.586309 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.586372 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.587378 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.587379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.587379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.587383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.587396 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.587399 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.587448 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.587507 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.587520 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:29:04.587733 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.585514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37605 (local address 127.13.15.129:44435)
0929 06:29:04.585712 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.585749 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.586218 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.586423 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.586425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.586433 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.587256 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.587256 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.587282 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.587306 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.587447 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.587449 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.587451 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.587499 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.587538 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260929 06:29:04.588184 18178 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37605
I20260929 06:29:04.588261 18178 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:04.588356 18178 heartbeater.cc:511] Master 127.13.15.190:37605 requested a full tablet report, sending...
I20260929 06:29:04.588790 17990 ts_manager.cc:194] Registered new tserver with Master: cad27348e45e43a796ed891855e1debd (127.13.15.129:43659)
I20260929 06:29:04.591509 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18050
I20260929 06:29:04.591600 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:04.591734 13374 external_mini_cluster.cc:1428] Setting key 3600d39d018f37cb4492bb6b5fdcfe7a
I20260929 06:29:04.592499 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--builtin_ntp_servers=127.13.15.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 06:29:04.669620 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:04.669809 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:04.669831 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:04.671027 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:04.671066 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:04.671092 18181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:04.672420 18181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:04.672626 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:04.672902 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:04.677654 18187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.677657 18186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.677703 18189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.677934 18181 server_base.cc:1034] running on GCE node
I20260929 06:29:04.678252 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:04.678480 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:04.679623 18181 hybrid_clock.cc:648] HybridClock initialized: now 1790663344679607 us; error 34 us; skew 500 ppm
I20260929 06:29:04.680701 18181 webserver.cc:460] Webserver started at http://127.13.15.130:45671/ using document root <none> and password file <none>
I20260929 06:29:04.680878 18181 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:04.680917 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:04.681001 18181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:04.681946 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "8216bbfdec46402387e56deec06b6db5"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "4af338ba046f53228e140845a1f2deee"
server_key_iv: "0987f681d9365c92cc9db44bc3df4047"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.682288 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "8216bbfdec46402387e56deec06b6db5"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "4af338ba046f53228e140845a1f2deee"
server_key_iv: "0987f681d9365c92cc9db44bc3df4047"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.683441 18181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:04.684073 18195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.684222 18181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.684283 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "8216bbfdec46402387e56deec06b6db5"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "4af338ba046f53228e140845a1f2deee"
server_key_iv: "0987f681d9365c92cc9db44bc3df4047"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.684329 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:04.687475 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:04.687659 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:04.687738 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:04.687891 18181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:04.688164 18181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:04.688217 18181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.688287 18181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:04.688315 18181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.693615 18181 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43405
I20260929 06:29:04.693658 18308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43405 every 8 connection(s)
I20260929 06:29:04.693961 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:04.696472 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18181
I20260929 06:29:04.696576 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:04.696640 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.694639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:47645 (local address 127.13.15.190:37605)
0929 06:29:04.694704 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.694706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.694926 (+ 220us) server_negotiation.cc:407] Connection header received
0929 06:29:04.695316 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.695317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.695321 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.695375 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.696383 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.696384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.696384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.696388 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.696401 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.696405 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.696449 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.696542 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.696559 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:29:04.696761 13374 external_mini_cluster.cc:1428] Setting key 60d912902e457908a43e226f8bd8f4c4
I20260929 06:29:04.696805 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.694634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37605 (local address 127.13.15.130:47645)
0929 06:29:04.694812 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.694847 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.695242 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.695420 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.695423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.695431 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.696228 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.696228 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.696252 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.696297 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.696450 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.696455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.696457 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.696538 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.696603 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260929 06:29:04.697466 18309 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37605
I20260929 06:29:04.697558 18309 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:04.697700 18309 heartbeater.cc:511] Master 127.13.15.190:37605 requested a full tablet report, sending...
I20260929 06:29:04.697762 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--builtin_ntp_servers=127.13.15.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 06:29:04.698045 17989 ts_manager.cc:194] Registered new tserver with Master: 8216bbfdec46402387e56deec06b6db5 (127.13.15.130:43405)
W20260929 06:29:04.774919 18312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:04.775110 18312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:04.775131 18312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:04.776260 18312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:04.776297 18312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:04.776324 18312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:04.777699 18312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:04.777920 18312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:04.778254 18312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:04.783049 18317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.783058 18318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:04.783049 18320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:04.783452 18312 server_base.cc:1034] running on GCE node
I20260929 06:29:04.783666 18312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:04.783854 18312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:04.785025 18312 hybrid_clock.cc:648] HybridClock initialized: now 1790663344785003 us; error 34 us; skew 500 ppm
I20260929 06:29:04.786302 18312 webserver.cc:460] Webserver started at http://127.13.15.131:33813/ using document root <none> and password file <none>
I20260929 06:29:04.786580 18312 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:04.786646 18312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:04.786783 18312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:04.787951 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "c403560134ab435a822cc357681c0ea9"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "c01f1f58a48d737496b5ee4335ad7a43"
server_key_iv: "5edfc245e2cac60e9132f7044fbd201b"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.788341 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "c403560134ab435a822cc357681c0ea9"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "c01f1f58a48d737496b5ee4335ad7a43"
server_key_iv: "5edfc245e2cac60e9132f7044fbd201b"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.789589 18312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:04.790310 18326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.790482 18312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:04.790580 18312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "c403560134ab435a822cc357681c0ea9"
format_stamp: "Formatted at 2026-09-29 06:29:04 on dist-test-slave-sm0p"
server_key: "c01f1f58a48d737496b5ee4335ad7a43"
server_key_iv: "5edfc245e2cac60e9132f7044fbd201b"
server_key_version: "encryptionkey@0"
I20260929 06:29:04.790663 18312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:04.793889 18312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:04.794194 18312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:04.794317 18312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:04.794521 18312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:04.794811 18312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:04.794863 18312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.794900 18312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:04.794926 18312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:04.800510 18312 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:39805
I20260929 06:29:04.800544 18439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:39805 every 8 connection(s)
I20260929 06:29:04.800858 18312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:04.802129 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18312
I20260929 06:29:04.802263 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:04.802408 13374 external_mini_cluster.cc:1428] Setting key ea3535728ea7595ebc9fc4691f875069
I20260929 06:29:04.804068 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.801610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:57103 (local address 127.13.15.190:37605)
0929 06:29:04.801670 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.801672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.801890 (+ 218us) server_negotiation.cc:407] Connection header received
0929 06:29:04.802592 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.802594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.802601 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.802664 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.803891 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.803892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.803892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.803896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.803910 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.803913 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.803959 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.804005 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.804015 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:29:04.804286 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.801598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37605 (local address 127.13.15.131:57103)
0929 06:29:04.801765 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.801803 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.802425 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.802750 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.802755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.802765 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.803762 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.803762 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.803788 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.803813 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.803957 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.803959 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.803962 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.804013 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.804057 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260929 06:29:04.804708 18440 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37605
I20260929 06:29:04.804783 18440 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:04.804885 18440 heartbeater.cc:511] Master 127.13.15.190:37605 requested a full tablet report, sending...
I20260929 06:29:04.805193 17989 ts_manager.cc:194] Registered new tserver with Master: c403560134ab435a822cc357681c0ea9 (127.13.15.131:39805)
I20260929 06:29:04.814301 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:29:04.816748 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.815619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.13.15.190:37605)
0929 06:29:04.815670 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.815672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.815790 (+ 118us) server_negotiation.cc:407] Connection header received
0929 06:29:04.815816 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.815817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.815821 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.815872 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.816487 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.816488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.816488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.816492 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.816505 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.816509 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.816554 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.816611 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.816622 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 06:29:04.818379 17989 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:29:04.819847 17989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:04.823506 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.821824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:39805 (local address 127.0.0.1:59184)
0929 06:29:04.822217 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.822226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.822276 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.822796 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.822797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.822800 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.823155 (+ 355us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.823156 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.823160 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.823167 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.823349 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.823356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.823359 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.823426 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.823438 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":51,"threads_started":1}
I20260929 06:29:04.823540 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.821939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.13.15.131:39805)
0929 06:29:04.822168 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.822173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.822322 (+ 149us) server_negotiation.cc:407] Connection header received
0929 06:29:04.822497 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.822502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.822513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.822959 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.823233 (+ 274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.823237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.823237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.823240 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.823270 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.823286 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.823351 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.823432 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.823449 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260929 06:29:04.823772 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.822046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.13.15.130:43405)
0929 06:29:04.822198 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.822201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.822209 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:04.822231 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.822232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.822240 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.822359 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.823354 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.823358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.823358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.823362 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.823388 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.823400 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.823667 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.823690 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.823705 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260929 06:29:04.823827 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.822014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.13.15.129:43659)
0929 06:29:04.822211 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:04.822215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:04.822523 (+ 308us) server_negotiation.cc:407] Connection header received
0929 06:29:04.822561 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:04.822564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:04.822573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:04.822643 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:04.823403 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:04.823412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:04.823413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:04.823418 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:04.823451 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:04.823478 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:04.823529 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:04.823738 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 06:29:04.823753 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260929 06:29:04.823606 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.821823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43405 (local address 127.0.0.1:54998)
0929 06:29:04.821964 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.822001 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.822052 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.822442 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.822449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.822455 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.822950 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.822951 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.822965 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.822988 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.823478 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.823479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.823480 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.823518 (+ 38us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.823531 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260929 06:29:04.823688 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:04.821830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:43659 (local address 127.0.0.1:46036)
0929 06:29:04.822424 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:04.822433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:04.822468 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:04.822919 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:04.822920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:04.822922 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:04.823300 (+ 378us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:04.823300 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:04.823304 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:04.823310 (+ 6us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:04.823553 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:04.823554 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:04.823559 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:29:04.823604 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 06:29:04.823618 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"spinlock_wait_cycles":384,"thread_start_us":79,"threads_started":1}
I20260929 06:29:04.824488 18374 tablet_service.cc:1467] Processing CreateTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 (DEFAULT_TABLE table=test-table [id=72da4d05b8b54e8da670f4e9ba0658d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.824519 18112 tablet_service.cc:1467] Processing CreateTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 (DEFAULT_TABLE table=test-table [id=72da4d05b8b54e8da670f4e9ba0658d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.824581 18243 tablet_service.cc:1467] Processing CreateTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 (DEFAULT_TABLE table=test-table [id=72da4d05b8b54e8da670f4e9ba0658d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:04.824862 18374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 620406cfe18043f98192e76c2b9b2fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.824862 18112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 620406cfe18043f98192e76c2b9b2fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.824928 18243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 620406cfe18043f98192e76c2b9b2fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:04.827587 18454 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Bootstrap starting.
I20260929 06:29:04.828184 18455 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Bootstrap starting.
I20260929 06:29:04.828176 18456 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Bootstrap starting.
I20260929 06:29:04.828306 18454 tablet_bootstrap.cc:654] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.828585 18454 log.cc:824] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.828816 18456 tablet_bootstrap.cc:654] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.828816 18455 tablet_bootstrap.cc:654] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:04.829133 18455 log.cc:824] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.829219 18456 log.cc:824] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:04.829324 18454 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: No bootstrap required, opened a new log
I20260929 06:29:04.829396 18454 ts_tablet_manager.cc:1397] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:04.829821 18455 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: No bootstrap required, opened a new log
I20260929 06:29:04.829886 18456 tablet_bootstrap.cc:492] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: No bootstrap required, opened a new log
I20260929 06:29:04.829905 18455 ts_tablet_manager.cc:1397] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:04.829949 18456 ts_tablet_manager.cc:1397] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:29:04.831916 18455 raft_consensus.cc:348] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.832062 18455 raft_consensus.cc:374] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.831967 18454 raft_consensus.cc:348] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.832119 18455 raft_consensus.cc:729] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad27348e45e43a796ed891855e1debd, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.832201 18454 raft_consensus.cc:374] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.832274 18454 raft_consensus.cc:729] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8216bbfdec46402387e56deec06b6db5, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.832274 18455 consensus_queue.cc:260] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.832412 18454 consensus_queue.cc:260] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.832511 18456 raft_consensus.cc:348] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.832610 18455 ts_tablet_manager.cc:1428] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 06:29:04.832647 18456 raft_consensus.cc:374] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:04.832667 18454 ts_tablet_manager.cc:1428] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:29:04.832692 18456 raft_consensus.cc:729] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c403560134ab435a822cc357681c0ea9, State: Initialized, Role: FOLLOWER
I20260929 06:29:04.832721 18178 heartbeater.cc:503] Master 127.13.15.190:37605 was elected leader, sending a full tablet report...
I20260929 06:29:04.832769 18309 heartbeater.cc:503] Master 127.13.15.190:37605 was elected leader, sending a full tablet report...
I20260929 06:29:04.832795 18456 consensus_queue.cc:260] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:04.833024 18456 ts_tablet_manager.cc:1428] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:29:04.833290 18440 heartbeater.cc:503] Master 127.13.15.190:37605 was elected leader, sending a full tablet report...
W20260929 06:29:04.835273 18179 tablet.cc:2369] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:29:04.944491 18310 tablet.cc:2369] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:29:05.051409 18441 tablet.cc:2369] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:05.079788 18462 raft_consensus.cc:482] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:05.079952 18462 raft_consensus.cc:504] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:05.080340 18462 leader_election.cc:288] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8216bbfdec46402387e56deec06b6db5 (127.13.15.130:43405), cad27348e45e43a796ed891855e1debd (127.13.15.129:43659)
I20260929 06:29:05.081558 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.080609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:53745 (local address 127.13.15.129:43659)
0929 06:29:05.080668 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.080671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.080682 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:29:05.080719 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.080720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.080726 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.080807 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.081347 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.081349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.081349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.081354 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.081368 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.081372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.081425 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.081482 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.081495 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:29:05.081558 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.080543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:43659 (local address 127.13.15.131:53745)
0929 06:29:05.080609 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.080621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.080668 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.080822 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.080822 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.080826 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:05.081250 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:05.081250 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:05.081257 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:05.081286 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:05.081432 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:05.081432 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:05.081433 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:29:05.081472 (+ 39us) client_negotiation.cc:240] Negotiation successful
0929 06:29:05.081486 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 06:29:05.081851 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.080606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:54029 (local address 127.13.15.130:43405)
0929 06:29:05.080661 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.080663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.080902 (+ 239us) server_negotiation.cc:407] Connection header received
0929 06:29:05.080925 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.080925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.080929 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.081008 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.081589 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.081590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.081590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.081595 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.081610 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.081615 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.081670 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.081777 (+ 107us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.081791 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 06:29:05.081903 18132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "620406cfe18043f98192e76c2b9b2fc7" candidate_uuid: "c403560134ab435a822cc357681c0ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad27348e45e43a796ed891855e1debd" is_pre_election: true
I20260929 06:29:05.082021 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.080548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43405 (local address 127.13.15.131:54029)
0929 06:29:05.080817 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.080826 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.080877 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.081026 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.081027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.081028 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:05.081455 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:29:05.081456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:05.081467 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:29:05.081478 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:05.081696 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:05.081696 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:05.081697 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:29:05.081941 (+ 244us) client_negotiation.cc:240] Negotiation successful
0929 06:29:05.081956 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":83,"thread_start_us":94,"threads_started":1}
I20260929 06:29:05.082119 18132 raft_consensus.cc:2454] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c403560134ab435a822cc357681c0ea9 in term 0.
I20260929 06:29:05.082362 18327 leader_election.cc:302] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c403560134ab435a822cc357681c0ea9, cad27348e45e43a796ed891855e1debd; no voters:
I20260929 06:29:05.082341 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "620406cfe18043f98192e76c2b9b2fc7" candidate_uuid: "c403560134ab435a822cc357681c0ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8216bbfdec46402387e56deec06b6db5" is_pre_election: true
I20260929 06:29:05.082472 18263 raft_consensus.cc:2454] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c403560134ab435a822cc357681c0ea9 in term 0.
I20260929 06:29:05.082486 18462 raft_consensus.cc:2781] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:05.082567 18462 raft_consensus.cc:482] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:05.082602 18462 raft_consensus.cc:3037] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.083280 18462 raft_consensus.cc:504] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:05.083413 18462 leader_election.cc:288] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [CANDIDATE]: Term 1 election: Requested vote from peers 8216bbfdec46402387e56deec06b6db5 (127.13.15.130:43405), cad27348e45e43a796ed891855e1debd (127.13.15.129:43659)
I20260929 06:29:05.083503 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "620406cfe18043f98192e76c2b9b2fc7" candidate_uuid: "c403560134ab435a822cc357681c0ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8216bbfdec46402387e56deec06b6db5"
I20260929 06:29:05.083549 18132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "620406cfe18043f98192e76c2b9b2fc7" candidate_uuid: "c403560134ab435a822cc357681c0ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad27348e45e43a796ed891855e1debd"
I20260929 06:29:05.083600 18263 raft_consensus.cc:3037] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.083642 18132 raft_consensus.cc:3037] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.084262 18263 raft_consensus.cc:2454] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c403560134ab435a822cc357681c0ea9 in term 1.
I20260929 06:29:05.084373 18132 raft_consensus.cc:2454] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c403560134ab435a822cc357681c0ea9 in term 1.
I20260929 06:29:05.084473 18329 leader_election.cc:302] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8216bbfdec46402387e56deec06b6db5, c403560134ab435a822cc357681c0ea9; no voters:
I20260929 06:29:05.084558 18462 raft_consensus.cc:2781] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:05.084676 18462 raft_consensus.cc:686] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 1 LEADER]: Becoming Leader. State: Replica: c403560134ab435a822cc357681c0ea9, State: Running, Role: LEADER
I20260929 06:29:05.084740 18462 consensus_queue.cc:237] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } }
I20260929 06:29:05.085624 17988 catalog_manager.cc:5495] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c403560134ab435a822cc357681c0ea9 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "c403560134ab435a822cc357681c0ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c403560134ab435a822cc357681c0ea9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 39805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 } health_report { overall_health: UNKNOWN } } }
I20260929 06:29:05.110742 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.109752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.13.15.131:39805)
0929 06:29:05.109794 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.109796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.109805 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:05.109875 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.109878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.109894 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.109960 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.110504 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.110506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.110506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.110511 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.110524 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.110528 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.110583 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.110642 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.110669 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:29:05.112493 18132 raft_consensus.cc:1261] T 620406cfe18043f98192e76c2b9b2fc7 P cad27348e45e43a796ed891855e1debd [term 1 FOLLOWER]: Refusing update from remote peer c403560134ab435a822cc357681c0ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:05.112493 18263 raft_consensus.cc:1261] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 1 FOLLOWER]: Refusing update from remote peer c403560134ab435a822cc357681c0ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:05.112875 18464 consensus_queue.cc:1035] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8216bbfdec46402387e56deec06b6db5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:05.113014 18462 consensus_queue.cc:1035] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cad27348e45e43a796ed891855e1debd" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:05.115319 18467 mvcc.cc:205] Tried to move back new op lower bound from 7334557061577998336 to 7334557061467758592. Current Snapshot: MvccSnapshot[applied={T|T < 7334557061577998336}]
I20260929 06:29:05.116005 18468 mvcc.cc:205] Tried to move back new op lower bound from 7334557061577998336 to 7334557061467758592. Current Snapshot: MvccSnapshot[applied={T|T < 7334557061577998336 or (T in {7334557061577998336})}]
I20260929 06:29:05.117872 17988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:05.117971 17988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:05.118793 17988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d59c8d9ec8a749a6bb3f1983e31e4568: Sending DeleteTablet for 3 replicas of tablet 620406cfe18043f98192e76c2b9b2fc7
I20260929 06:29:05.119091 18243 tablet_service.cc:1514] Processing DeleteTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:05 UTC) from {username='slave'} at 127.0.0.1:54998
I20260929 06:29:05.119102 18374 tablet_service.cc:1514] Processing DeleteTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:05 UTC) from {username='slave'} at 127.0.0.1:59184
I20260929 06:29:05.119285 18112 tablet_service.cc:1514] Processing DeleteTablet for tablet 620406cfe18043f98192e76c2b9b2fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:05 UTC) from {username='slave'} at 127.0.0.1:46036
I20260929 06:29:05.119308 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18050
I20260929 06:29:05.119985 18478 tablet_replica.cc:330] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: stopping tablet replica
I20260929 06:29:05.120065 18478 raft_consensus.cc:2229] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:05.120179 18478 raft_consensus.cc:2258] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:05.120596 18478 ts_tablet_manager.cc:1905] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:05.122066 18477 tablet_replica.cc:330] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: stopping tablet replica
I20260929 06:29:05.122193 18478 ts_tablet_manager.cc:1918] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:05.122236 18477 raft_consensus.cc:2229] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:05.122337 18478 log.cc:1196] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/620406cfe18043f98192e76c2b9b2fc7
I20260929 06:29:05.122375 18477 raft_consensus.cc:2258] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:05.122653 18478 ts_tablet_manager.cc:1939] T 620406cfe18043f98192e76c2b9b2fc7 P 8216bbfdec46402387e56deec06b6db5: Deleting consensus metadata
I20260929 06:29:05.122819 18477 ts_tablet_manager.cc:1905] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:05.123381 17976 catalog_manager.cc:4841] TS 8216bbfdec46402387e56deec06b6db5 (127.13.15.130:43405): tablet 620406cfe18043f98192e76c2b9b2fc7 (table test-table [id=72da4d05b8b54e8da670f4e9ba0658d5]) successfully deleted
I20260929 06:29:05.124363 18477 ts_tablet_manager.cc:1918] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:05.124430 18477 log.cc:1196] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/620406cfe18043f98192e76c2b9b2fc7
I20260929 06:29:05.124755 18477 ts_tablet_manager.cc:1939] T 620406cfe18043f98192e76c2b9b2fc7 P c403560134ab435a822cc357681c0ea9: Deleting consensus metadata
W20260929 06:29:05.124878 17974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:43659 (error 108)
I20260929 06:29:05.125108 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18181
I20260929 06:29:05.125159 17975 catalog_manager.cc:4841] TS c403560134ab435a822cc357681c0ea9 (127.13.15.131:39805): tablet 620406cfe18043f98192e76c2b9b2fc7 (table test-table [id=72da4d05b8b54e8da670f4e9ba0658d5]) successfully deleted
I20260929 06:29:05.125643 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.125177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:43659 (local address 127.0.0.1:46050)
0929 06:29:05.125555 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.125592 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359}
W20260929 06:29:05.125847 17974 catalog_manager.cc:4584] TS cad27348e45e43a796ed891855e1debd (127.13.15.129:43659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 620406cfe18043f98192e76c2b9b2fc7: Network error: Client connection negotiation failed: client connection to 127.13.15.129:43659: connect: Connection refused (error 111)
I20260929 06:29:05.129901 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18312
I20260929 06:29:05.134184 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 17959
2026-09-29T06:29:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (822 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 06:29:05.153024 13374 mini_oidc.cc:152] Starting JWKS server
I20260929 06:29:05.153985 13374 webserver.cc:460] Webserver started at https://127.0.0.1:32945/ using document root <none> and password file <none>
I20260929 06:29:05.154086 13374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 06:29:05.154289 13374 webserver.cc:460] Webserver started at http://127.0.0.1:39759/ using document root <none> and password file <none>
2026-09-29T06:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:05Z Disabled control of system clock
I20260929 06:29:05.161206 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34547
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:36161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34547
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32945/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/testchainca.cert with env {}
W20260929 06:29:05.238933 18485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:05.239136 18485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:05.239158 18485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:05.240378 18485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:05.240420 18485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:05.240433 18485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:05.240446 18485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:05.240458 18485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 06:29:05.240470 18485 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 06:29:05.240482 18485 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32945/jwks.json
I20260929 06:29:05.242048 18485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32945/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:05.242319 18485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:05.242574 18485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:29:05.247501 18485 server_base.cc:1034] running on GCE node
W20260929 06:29:05.247485 18490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.247484 18493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.247495 18491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:05.247934 18485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:05.248225 18485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:05.249384 18485 hybrid_clock.cc:648] HybridClock initialized: now 1790663345249369 us; error 44 us; skew 500 ppm
I20260929 06:29:05.250594 18485 webserver.cc:460] Webserver started at http://127.13.15.190:35485/ using document root <none> and password file <none>
I20260929 06:29:05.250825 18485 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:05.250890 18485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:05.251035 18485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:05.252009 18485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "3f1660d8332e4d5a83efa4fd76adc139"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "bbf8fe96302efdd58af78fa73290e24f"
server_key_iv: "20707c56ce26df6c052516841e917054"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.252346 18485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "3f1660d8332e4d5a83efa4fd76adc139"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "bbf8fe96302efdd58af78fa73290e24f"
server_key_iv: "20707c56ce26df6c052516841e917054"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.253568 18485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:05.254297 18499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:05.254463 18485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:05.254577 18485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "3f1660d8332e4d5a83efa4fd76adc139"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "bbf8fe96302efdd58af78fa73290e24f"
server_key_iv: "20707c56ce26df6c052516841e917054"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.254655 18485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:05.281348 18485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:05.281708 18485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:05.281864 18485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:05.285712 18485 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34547
I20260929 06:29:05.285748 18551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34547 every 8 connection(s)
I20260929 06:29:05.286080 18485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:05.286996 18552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:05.289178 18552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139: Bootstrap starting.
I20260929 06:29:05.289743 18552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:05.290031 18552 log.cc:824] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:05.290757 18552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139: No bootstrap required, opened a new log
I20260929 06:29:05.292296 18552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } }
I20260929 06:29:05.292429 18552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:05.292451 18552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1660d8332e4d5a83efa4fd76adc139, State: Initialized, Role: FOLLOWER
I20260929 06:29:05.292600 18552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } }
I20260929 06:29:05.292675 18552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:05.292709 18552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:05.292776 18552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.293282 18552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } }
I20260929 06:29:05.293390 18552 leader_election.cc:302] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f1660d8332e4d5a83efa4fd76adc139; no voters:
I20260929 06:29:05.293567 18552 leader_election.cc:288] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:05.293639 18555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:05.293798 18555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [term 1 LEADER]: Becoming Leader. State: Replica: 3f1660d8332e4d5a83efa4fd76adc139, State: Running, Role: LEADER
I20260929 06:29:05.293812 18552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:05.293900 18555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } }
I20260929 06:29:05.294266 18557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f1660d8332e4d5a83efa4fd76adc139. Latest consensus state: current_term: 1 leader_uuid: "3f1660d8332e4d5a83efa4fd76adc139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } } }
I20260929 06:29:05.294363 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f1660d8332e4d5a83efa4fd76adc139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1660d8332e4d5a83efa4fd76adc139" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34547 } } }
I20260929 06:29:05.294378 18557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.294426 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.294611 18563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:05.295156 18563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:05.295810 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18485
I20260929 06:29:05.295912 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:05.296089 13374 external_mini_cluster.cc:1428] Setting key 91d2d4bc1a04d7ffa0dda58d18bac865
I20260929 06:29:05.297241 18563 catalog_manager.cc:1347] Generated new cluster ID: 4d4606df0aef480085d7e1f1ae74012d
I20260929 06:29:05.297299 18563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:05.301779 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.296606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.13.15.190:34547)
0929 06:29:05.296781 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.296792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.296822 (+ 30us) server_negotiation.cc:407] Connection header received
0929 06:29:05.296907 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.296912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.296961 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.297102 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.297323 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.298674 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.299751 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.300344 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.300448 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.301131 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.301135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.301136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.301319 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.301354 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.301366 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.301424 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.301534 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.301602 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260929 06:29:05.302541 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:29:05.306067 18563 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:05.306520 18563 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:05.310726 18563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f1660d8332e4d5a83efa4fd76adc139: Generated new TSK 0
I20260929 06:29:05.310873 18563 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 06:29:05.334447 18582 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 06:29:05.334537 18582 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 06:29:05.334739 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.319841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.13.15.190:34547)
0929 06:29:05.319881 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.319883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.320005 (+ 122us) server_negotiation.cc:407] Connection header received
0929 06:29:05.320100 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.320100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.320109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.320162 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0929 06:29:05.320342 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.320805 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.321849 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.322014 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.322065 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.322300 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 06:29:05.334547 (+ 12247us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 06:29:05.334686 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40894: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 06:29:05.334825 18574 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40894: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 06:29:05.335156 18581 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:05.319764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34547 (local address 127.0.0.1:40894)
0929 06:29:05.319935 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.319946 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.320019 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.320186 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.320186 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.320188 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 06:29:05.320279 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.320281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.320875 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.320876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.321778 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.321780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.322091 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.322092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.322175 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.322197 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 06:29:05.334711 (+ 12514us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 06:29:05.334843 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.15.190:34547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.15.190:34547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 06:29:05.335601 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18485
2026-09-29T06:29:05Z chronyd exiting
I20260929 06:29:05.371330 13374 test_util.cc:175] -----------------------------------------------
I20260929 06:29:05.371419 13374 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790663323716959-13374-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 06:29:05.372125 13374 mini_oidc.cc:152] Starting JWKS server
I20260929 06:29:05.372756 13374 webserver.cc:460] Webserver started at https://127.0.0.1:43715/ using document root <none> and password file <none>
I20260929 06:29:05.372817 13374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 06:29:05.372972 13374 webserver.cc:460] Webserver started at http://127.0.0.1:36827/ using document root <none> and password file <none>
2026-09-29T06:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:05Z Disabled control of system clock
I20260929 06:29:05.380155 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:46565
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:46565
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43715/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/testchainca.cert with env {}
W20260929 06:29:05.455864 18588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:05.456094 18588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:05.456115 18588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:05.457252 18588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:05.457290 18588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:05.457304 18588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:05.457315 18588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:05.457327 18588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 06:29:05.457338 18588 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 06:29:05.457350 18588 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43715/jwks.json
I20260929 06:29:05.459154 18588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:46565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43715/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:05.459424 18588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:05.459650 18588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:05.464386 18593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.464391 18594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.464465 18596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:05.464488 18588 server_base.cc:1034] running on GCE node
I20260929 06:29:05.464812 18588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:05.465128 18588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:05.466286 18588 hybrid_clock.cc:648] HybridClock initialized: now 1790663345466279 us; error 37 us; skew 500 ppm
I20260929 06:29:05.467648 18588 webserver.cc:460] Webserver started at http://127.13.15.190:34021/ using document root <none> and password file <none>
I20260929 06:29:05.467890 18588 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:05.467955 18588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:05.468097 18588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:05.469125 18588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "e07824980e3744e6a4e73ded7c4aceee"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "9c380fe102e85fb6472451a12acbefe2"
server_key_iv: "2ecef3db69060a72317fec2af9d10675"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.469511 18588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "e07824980e3744e6a4e73ded7c4aceee"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "9c380fe102e85fb6472451a12acbefe2"
server_key_iv: "2ecef3db69060a72317fec2af9d10675"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.470875 18588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:05.471556 18602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:05.471714 18588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:05.471815 18588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "e07824980e3744e6a4e73ded7c4aceee"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "9c380fe102e85fb6472451a12acbefe2"
server_key_iv: "2ecef3db69060a72317fec2af9d10675"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.471886 18588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:05.504190 18588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:05.504552 18588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:05.504721 18588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:05.508809 18588 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:46565
I20260929 06:29:05.508844 18654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:46565 every 8 connection(s)
I20260929 06:29:05.509227 18588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:05.509887 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:05.512552 18655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee: Bootstrap starting.
I20260929 06:29:05.513077 18655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:05.513302 18655 log.cc:824] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:05.513890 18655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee: No bootstrap required, opened a new log
I20260929 06:29:05.514828 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18588
I20260929 06:29:05.514983 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:05.515172 13374 external_mini_cluster.cc:1428] Setting key b61225cb28c2759c6d0e7b8b00e1c5c8
I20260929 06:29:05.516182 18655 raft_consensus.cc:348] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } }
I20260929 06:29:05.516381 18655 raft_consensus.cc:374] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:05.516422 18655 raft_consensus.cc:729] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07824980e3744e6a4e73ded7c4aceee, State: Initialized, Role: FOLLOWER
I20260929 06:29:05.516568 18655 consensus_queue.cc:260] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } }
I20260929 06:29:05.516652 18655 raft_consensus.cc:388] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:05.516675 18655 raft_consensus.cc:482] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:05.516737 18655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.517450 18655 raft_consensus.cc:504] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } }
I20260929 06:29:05.517544 18655 leader_election.cc:302] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07824980e3744e6a4e73ded7c4aceee; no voters:
I20260929 06:29:05.517736 18655 leader_election.cc:288] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:05.517771 18660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:05.517890 18660 raft_consensus.cc:686] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [term 1 LEADER]: Becoming Leader. State: Replica: e07824980e3744e6a4e73ded7c4aceee, State: Running, Role: LEADER
I20260929 06:29:05.518036 18655 sys_catalog.cc:564] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:05.518020 18660 consensus_queue.cc:237] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } }
I20260929 06:29:05.518497 18661 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e07824980e3744e6a4e73ded7c4aceee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } } }
I20260929 06:29:05.518584 18661 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.518817 18662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e07824980e3744e6a4e73ded7c4aceee. Latest consensus state: current_term: 1 leader_uuid: "e07824980e3744e6a4e73ded7c4aceee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07824980e3744e6a4e73ded7c4aceee" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46565 } } }
I20260929 06:29:05.519119 18662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.519273 18667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:05.519834 18667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:05.521323 18667 catalog_manager.cc:1347] Generated new cluster ID: f799f258eec84801857e3e80bab371e8
I20260929 06:29:05.521435 18667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:05.521683 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.515655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.13.15.190:46565)
0929 06:29:05.515858 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.515865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.515889 (+ 24us) server_negotiation.cc:407] Connection header received
0929 06:29:05.516129 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.516134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.516208 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.516333 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.516590 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.518058 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.519235 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.519886 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.519999 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.520954 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.520958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.520958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.521149 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.521183 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.521193 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.521280 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.521419 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.521494 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260929 06:29:05.522495 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:29:05.528081 18667 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:05.528548 18667 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:05.533419 18667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e07824980e3744e6a4e73ded7c4aceee: Generated new TSK 0
I20260929 06:29:05.533589 18667 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 06:29:05.561246 18685 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 06:29:05.561321 18685 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 06:29:05.561579 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.539633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.13.15.190:46565)
0929 06:29:05.539686 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.539688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.539746 (+ 58us) server_negotiation.cc:407] Connection header received
0929 06:29:05.539825 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.539826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.539835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.539883 (+ 48us) server_negotiation.cc:226] Negotiated authn=JWT
0929 06:29:05.540025 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.540500 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.541632 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.541810 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.541857 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.542024 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 06:29:05.561383 (+ 19359us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 06:29:05.561520 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 06:29:05.561645 18659 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 06:29:05.561692 18684 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:05.539542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:46565 (local address 127.0.0.1:59272)
0929 06:29:05.539689 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.539700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.539763 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.539892 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.539893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.539913 (+ 20us) client_negotiation.cc:189] Negotiated authn=JWT
0929 06:29:05.539966 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.539968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.540588 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.540589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.541556 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.541558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.541880 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.541881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.541937 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.541942 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 06:29:05.561479 (+ 19537us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 06:29:05.561579 (+ 100us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.15.190:46565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260929 06:29:05.561993 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18588
2026-09-29T06:29:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 06:29:05.599074 13374 mini_oidc.cc:152] Starting JWKS server
I20260929 06:29:05.599740 13374 webserver.cc:460] Webserver started at https://127.0.0.1:40485/ using document root <none> and password file <none>
I20260929 06:29:05.599781 13374 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 06:29:05.599951 13374 webserver.cc:460] Webserver started at http://127.0.0.1:42627/ using document root <none> and password file <none>
2026-09-29T06:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:05Z Disabled control of system clock
I20260929 06:29:05.607151 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44787
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44787
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40485/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 06:29:05.684648 18691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:05.684894 18691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:05.684931 18691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:05.686082 18691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:05.686151 18691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:05.686203 18691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:05.686231 18691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:05.686260 18691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 06:29:05.686285 18691 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 06:29:05.686314 18691 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40485/jwks.json
I20260929 06:29:05.687618 18691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40485/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:05.687875 18691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:05.688140 18691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:05.692831 18697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.692842 18696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:05.693027 18691 server_base.cc:1034] running on GCE node
W20260929 06:29:05.692863 18699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:05.693320 18691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:05.693595 18691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:05.694759 18691 hybrid_clock.cc:648] HybridClock initialized: now 1790663345694738 us; error 34 us; skew 500 ppm
I20260929 06:29:05.696012 18691 webserver.cc:460] Webserver started at http://127.13.15.190:40337/ using document root <none> and password file <none>
I20260929 06:29:05.696249 18691 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:05.696314 18691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:05.696457 18691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:05.697491 18691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "7fe7633825024ebf8286ef1b71c63469"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "27d40aa3ed64246bbc7b444aae091b36"
server_key_iv: "353495df83a530e6818007c09786235d"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.697852 18691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "7fe7633825024ebf8286ef1b71c63469"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "27d40aa3ed64246bbc7b444aae091b36"
server_key_iv: "353495df83a530e6818007c09786235d"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.699162 18691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:05.699877 18705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:05.700029 18691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:05.700124 18691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "7fe7633825024ebf8286ef1b71c63469"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "27d40aa3ed64246bbc7b444aae091b36"
server_key_iv: "353495df83a530e6818007c09786235d"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.700205 18691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:05.720352 18691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:05.720669 18691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:05.720818 18691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:05.724677 18691 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44787
I20260929 06:29:05.724741 18757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44787 every 8 connection(s)
I20260929 06:29:05.725075 18691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:05.725744 18758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:05.728353 18758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469: Bootstrap starting.
I20260929 06:29:05.728900 18758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:05.729174 18758 log.cc:824] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:05.729781 18758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469: No bootstrap required, opened a new log
I20260929 06:29:05.731340 18758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } }
I20260929 06:29:05.731426 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18691
I20260929 06:29:05.731515 18758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:05.731552 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:05.731555 18758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe7633825024ebf8286ef1b71c63469, State: Initialized, Role: FOLLOWER
I20260929 06:29:05.731750 13374 external_mini_cluster.cc:1428] Setting key 0dfe2089c74e0e4196516e608423311c
I20260929 06:29:05.731730 18758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } }
I20260929 06:29:05.731814 18758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:05.731855 18758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:05.731906 18758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.732610 18758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } }
I20260929 06:29:05.732718 18758 leader_election.cc:302] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fe7633825024ebf8286ef1b71c63469; no voters:
I20260929 06:29:05.732870 18758 leader_election.cc:288] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:05.732914 18763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:05.733052 18763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [term 1 LEADER]: Becoming Leader. State: Replica: 7fe7633825024ebf8286ef1b71c63469, State: Running, Role: LEADER
I20260929 06:29:05.733117 18758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:05.733155 18763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } }
I20260929 06:29:05.733517 18763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fe7633825024ebf8286ef1b71c63469. Latest consensus state: current_term: 1 leader_uuid: "7fe7633825024ebf8286ef1b71c63469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } } }
I20260929 06:29:05.733593 18763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.733738 18765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fe7633825024ebf8286ef1b71c63469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe7633825024ebf8286ef1b71c63469" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44787 } } }
I20260929 06:29:05.733791 18765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.733902 18771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:05.734848 18771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:05.736377 18771 catalog_manager.cc:1347] Generated new cluster ID: f65208ab853c4d48ab7b64c286df63c6
I20260929 06:29:05.736433 18771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:05.738457 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.732195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.13.15.190:44787)
0929 06:29:05.732385 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.732394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.732426 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:29:05.732548 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.732556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.732636 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.732780 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.733077 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.734923 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.736035 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.736627 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.736741 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.737596 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.737599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.737599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.737789 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.737826 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.737839 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.737896 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.738162 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.738237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260929 06:29:05.739444 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:29:05.747110 18771 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:05.747608 18771 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:05.752149 18771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fe7633825024ebf8286ef1b71c63469: Generated new TSK 0
I20260929 06:29:05.752310 18771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:29:05.765944 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.762276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38914 (local address 127.13.15.190:44787)
0929 06:29:05.762337 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.762339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.762496 (+ 157us) server_negotiation.cc:407] Connection header received
0929 06:29:05.762573 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.762573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.762583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.762698 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0929 06:29:05.762812 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.763292 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.764421 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.764624 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.764673 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.764855 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 06:29:05.765732 (+ 877us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40485/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 06:29:05.765860 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40485/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260929 06:29:05.766002 18762 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40485/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 06:29:05.766163 18787 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:05.762237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44787 (local address 127.0.0.1:38914)
0929 06:29:05.762425 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.762437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.762500 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.762699 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.762700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.762703 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0929 06:29:05.762751 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.762753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.763371 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.763372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.764312 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.764315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.764698 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.764699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.764757 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.764763 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 06:29:05.765833 (+ 1070us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40485/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 06:29:05.765924 (+ 91us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.15.190:44787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40485/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260929 06:29:05.766372 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18691
E20260929 06:29:05.776862 18788 webserver.cc:533] Webserver: sslize failed:
E20260929 06:29:05.776975 18788 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T06:29:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (204 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T06:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:05Z Disabled control of system clock
I20260929 06:29:05.812960 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35451
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:35451
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/testchainca.cert with env {}
W20260929 06:29:05.891894 18792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:05.892099 18792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:05.892122 18792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:05.893191 18792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:05.893222 18792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:05.893235 18792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:05.893249 18792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:05.893260 18792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 06:29:05.893272 18792 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 06:29:05.893285 18792 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 06:29:05.894665 18792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:35451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:05.894845 18792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:05.894994 18792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:29:05.899811 18792 server_base.cc:1034] running on GCE node
W20260929 06:29:05.899780 18798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.899812 18797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:05.899780 18800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:05.900245 18792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:05.900537 18792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:05.901701 18792 hybrid_clock.cc:648] HybridClock initialized: now 1790663345901688 us; error 31 us; skew 500 ppm
I20260929 06:29:05.902941 18792 webserver.cc:460] Webserver started at http://127.13.15.190:38645/ using document root <none> and password file <none>
I20260929 06:29:05.903183 18792 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:05.903250 18792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:05.903396 18792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:05.904422 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "391c0ff62c1d4fbb98d20c42f635f9e7"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "36005a779cf5e3e7c9cc1e4a1de14a55"
server_key_iv: "3237c688774a8b51dab3c4f205b93272"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.904793 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "391c0ff62c1d4fbb98d20c42f635f9e7"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "36005a779cf5e3e7c9cc1e4a1de14a55"
server_key_iv: "3237c688774a8b51dab3c4f205b93272"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.906095 18792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:05.906840 18806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:05.907028 18792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:05.907145 18792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "391c0ff62c1d4fbb98d20c42f635f9e7"
format_stamp: "Formatted at 2026-09-29 06:29:05 on dist-test-slave-sm0p"
server_key: "36005a779cf5e3e7c9cc1e4a1de14a55"
server_key_iv: "3237c688774a8b51dab3c4f205b93272"
server_key_version: "encryptionkey@0"
I20260929 06:29:05.907222 18792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:05.940824 18792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:05.941202 18792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:05.941366 18792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:05.945058 18792 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:35451
I20260929 06:29:05.945097 18858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:35451 every 8 connection(s)
I20260929 06:29:05.945453 18792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:05.946496 18859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:05.947405 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18792
I20260929 06:29:05.947530 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:05.947710 13374 external_mini_cluster.cc:1428] Setting key 1c2a705db6dfc9cde3e6346037cb607f
I20260929 06:29:05.949323 18859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7: Bootstrap starting.
I20260929 06:29:05.950030 18859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:05.950361 18859 log.cc:824] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:05.951093 18859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7: No bootstrap required, opened a new log
I20260929 06:29:05.953277 18859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } }
I20260929 06:29:05.953449 18859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:05.953486 18859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 391c0ff62c1d4fbb98d20c42f635f9e7, State: Initialized, Role: FOLLOWER
I20260929 06:29:05.953629 18859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } }
I20260929 06:29:05.953722 18859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:05.953751 18859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:05.953799 18859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:05.953806 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.948239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.13.15.190:35451)
0929 06:29:05.948464 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.948474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.948506 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:29:05.949082 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.949091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.949172 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.949320 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:05.949535 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.950882 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.951655 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.952265 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.952411 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.953039 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:05.953044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:05.953044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:29:05.953245 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:05.953286 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:29:05.953302 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:05.953369 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:05.953426 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:29:05.953601 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260929 06:29:05.954392 18859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } }
I20260929 06:29:05.954521 18859 leader_election.cc:302] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 391c0ff62c1d4fbb98d20c42f635f9e7; no voters:
I20260929 06:29:05.954725 18859 leader_election.cc:288] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:05.954736 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:29:05.954752 18864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:05.954860 18864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [term 1 LEADER]: Becoming Leader. State: Replica: 391c0ff62c1d4fbb98d20c42f635f9e7, State: Running, Role: LEADER
I20260929 06:29:05.954937 18864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } }
I20260929 06:29:05.954972 18859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:05.955256 18864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 391c0ff62c1d4fbb98d20c42f635f9e7. Latest consensus state: current_term: 1 leader_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } } }
I20260929 06:29:05.955327 18864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.955536 18870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:05.955273 18865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391c0ff62c1d4fbb98d20c42f635f9e7" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35451 } } }
I20260929 06:29:05.955941 18865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:05.956012 18870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:05.957442 18870 catalog_manager.cc:1347] Generated new cluster ID: 829c242fec43417aaca8767566c0f452
I20260929 06:29:05.957492 18870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:05.973459 18870 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:05.973937 18870 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:05.981948 18870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 391c0ff62c1d4fbb98d20c42f635f9e7: Generated new TSK 0
I20260929 06:29:05.982131 18870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:29:06.414882 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:05.986699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.13.15.190:35451)
0929 06:29:05.986744 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:05.986746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:05.986842 (+ 96us) server_negotiation.cc:407] Connection header received
0929 06:29:05.986923 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:05.986923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:05.986938 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:05.986995 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0929 06:29:05.987135 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.987599 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.988703 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.988877 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.988924 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.989124 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 06:29:06.414671 (+425547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 06:29:06.414822 (+ 151us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47730: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 06:29:06.414973 18862 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47730: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 06:29:06.415040 18888 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:05.986653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35451 (local address 127.0.0.1:47730)
0929 06:29:05.986773 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:05.986784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:05.986859 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:05.987013 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:05.987014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:05.987016 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 06:29:05.987075 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.987077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.987675 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.987676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.988623 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:05.988626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:05.988950 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:05.988951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:05.989030 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:05.989036 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 06:29:06.414815 (+425779us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 06:29:06.414938 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.15.190:35451: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260929 06:29:06.415380 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18792
2026-09-29T06:29:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (628 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:06 dist-test-slave-sm0p krb5kdc[18900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:06 dist-test-slave-sm0p krb5kdc[18900](info): set up 2 sockets
Sep 29 06:29:06 dist-test-slave-sm0p krb5kdc[18900](info): commencing operation
krb5kdc: starting...
W20260929 06:29:07.602434 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:07 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:07.716631 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44539
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:07.794062 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:07.794297 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:07.794333 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:07.795476 18913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:07.795540 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:07.795565 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:07.795591 18913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:07.795630 18913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:07.797114 18913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:07.797380 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:07.797591 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:07.802321 18921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:07.802368 18918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:07.802338 18919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:07.805263 18913 server_base.cc:1034] running on GCE node
I20260929 06:29:07.805588 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:07.805984 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:07.807158 18913 hybrid_clock.cc:648] HybridClock initialized: now 1790663347807130 us; error 41 us; skew 500 ppm
Sep 29 06:29:07 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663347, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:07.813971 18913 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:07.814469 18913 webserver.cc:460] Webserver started at http://127.13.15.190:44457/ using document root <none> and password file <none>
I20260929 06:29:07.814704 18913 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:07.814774 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:07.814909 18913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:07.815974 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "684d9c11ddb748b7aeb4858ff02c20aa"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "9128df83c82ab160a31e612330b57d6b"
server_key_iv: "52316cfb2fb7a97b40ec2961e652687c"
server_key_version: "encryptionkey@0"
I20260929 06:29:07.816346 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "684d9c11ddb748b7aeb4858ff02c20aa"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "9128df83c82ab160a31e612330b57d6b"
server_key_iv: "52316cfb2fb7a97b40ec2961e652687c"
server_key_version: "encryptionkey@0"
I20260929 06:29:07.817588 18913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:07.818349 18928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:07.818564 18913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:07.818665 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "684d9c11ddb748b7aeb4858ff02c20aa"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "9128df83c82ab160a31e612330b57d6b"
server_key_iv: "52316cfb2fb7a97b40ec2961e652687c"
server_key_version: "encryptionkey@0"
I20260929 06:29:07.818742 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:07.836272 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:07.838546 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:07.838727 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:07.842541 18913 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44539
I20260929 06:29:07.842577 18980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44539 every 8 connection(s)
I20260929 06:29:07.842937 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:07.843734 18981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:07.845932 18981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: Bootstrap starting.
I20260929 06:29:07.846519 18981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:07.846808 18981 log.cc:824] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:07.847476 18981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: No bootstrap required, opened a new log
I20260929 06:29:07.848961 18981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } }
I20260929 06:29:07.849092 18981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:07.849159 18981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 684d9c11ddb748b7aeb4858ff02c20aa, State: Initialized, Role: FOLLOWER
I20260929 06:29:07.849308 18981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } }
I20260929 06:29:07.849367 18981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:07.849404 18981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:07.849453 18981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:07.849984 18981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } }
I20260929 06:29:07.850090 18981 leader_election.cc:302] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 684d9c11ddb748b7aeb4858ff02c20aa; no voters:
I20260929 06:29:07.850303 18981 leader_election.cc:288] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:07.850343 18984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:07.850567 18981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:07.850672 18984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [term 1 LEADER]: Becoming Leader. State: Replica: 684d9c11ddb748b7aeb4858ff02c20aa, State: Running, Role: LEADER
I20260929 06:29:07.850972 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 18913
I20260929 06:29:07.851084 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:07.851251 13374 external_mini_cluster.cc:1428] Setting key bb02f5a9e2009b4a89344b091a9f5741
I20260929 06:29:07.850759 18984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } }
I20260929 06:29:07.852003 18985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } } }
I20260929 06:29:07.852015 18986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 684d9c11ddb748b7aeb4858ff02c20aa. Latest consensus state: current_term: 1 leader_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684d9c11ddb748b7aeb4858ff02c20aa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44539 } } }
I20260929 06:29:07.852210 18985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:07.852213 18986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa [sys.catalog]: This master's current role is: LEADER
W20260929 06:29:07.852308 19000 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:29:07.852355 19000 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:29:07.852420 19002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:07.852583 19002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:07.854233 19002 catalog_manager.cc:1347] Generated new cluster ID: 927e2e09f4954d4c922e723feccccdf2
I20260929 06:29:07.854283 19002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:29:07 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:07.863125 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:07.851707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.13.15.190:44539)
0929 06:29:07.852374 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:07.852383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:07.852411 (+ 28us) server_negotiation.cc:407] Connection header received
0929 06:29:07.852548 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:07.852555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:07.852612 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:07.852754 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:07.853513 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:07.854935 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:07.856063 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:07.856655 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:07.856739 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:07.859484 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:07.859489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:07.859489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:07.859502 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:07.861491 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:07.861879 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:07.861883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:07.861883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:07.861991 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:07.862209 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:07.862210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:07.862210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:07.862524 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:07.862690 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:07.862756 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 06:29:07.862887 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":60,"threads_started":1}
I20260929 06:29:07.867681 19002 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:07.868381 19002 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:07.878471 19002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 684d9c11ddb748b7aeb4858ff02c20aa: Generated new TSK 0
I20260929 06:29:07.878665 19002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:29:07.905148 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:07.985759 19007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:07.985962 19007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:07.985986 19007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:07.987207 19007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:07.987250 19007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:07.987277 19007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:07.988773 19007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:07.988991 19007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:07.989235 19007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:07.994369 19015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:07.994417 19007 server_base.cc:1034] running on GCE node
W20260929 06:29:07.994345 19012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:07.994345 19013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:07.994802 19007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:07.995043 19007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:07.996201 19007 hybrid_clock.cc:648] HybridClock initialized: now 1790663347996194 us; error 30 us; skew 500 ppm
Sep 29 06:29:07 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663347, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:07.998720 19007 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:29:07.999190 19007 webserver.cc:460] Webserver started at http://127.13.15.129:41547/ using document root <none> and password file <none>
I20260929 06:29:07.999419 19007 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:07.999488 19007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:07.999641 19007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:08.000672 19007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "ca2e90f5e4dc4f638ece286fba4021d8"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "3e29e01def1c9305b5d9cebf0288c8a2"
server_key_iv: "837ae11b150af8d54bb0246b590323f2"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.001039 19007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "ca2e90f5e4dc4f638ece286fba4021d8"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "3e29e01def1c9305b5d9cebf0288c8a2"
server_key_iv: "837ae11b150af8d54bb0246b590323f2"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.002421 19007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:08.003099 19022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.003266 19007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:08.003367 19007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "ca2e90f5e4dc4f638ece286fba4021d8"
format_stamp: "Formatted at 2026-09-29 06:29:07 on dist-test-slave-sm0p"
server_key: "3e29e01def1c9305b5d9cebf0288c8a2"
server_key_iv: "837ae11b150af8d54bb0246b590323f2"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.003439 19007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:08.020118 19007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:08.022482 19007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:08.022648 19007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:08.022902 19007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:08.023206 19007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:08.023258 19007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.023298 19007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:08.023324 19007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.028962 19007 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:33991
I20260929 06:29:08.029013 19135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:33991 every 8 connection(s)
I20260929 06:29:08.029367 19007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:08.029868 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19007
I20260929 06:29:08.029975 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:08.030187 13374 external_mini_cluster.cc:1428] Setting key 1403ca37c536b92f9ff3e49528a2e288
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663347, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:29:08.040210 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.030697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:52319 (local address 127.13.15.190:44539)
0929 06:29:08.030742 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.030744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.030965 (+ 221us) server_negotiation.cc:407] Connection header received
0929 06:29:08.032320 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.032321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.032330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.032391 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.033035 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.033834 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.034922 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.035288 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.035373 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.037447 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.037448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.037449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.037457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.039096 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.039436 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.039437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.039437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.039485 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.039703 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.039704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.039704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.039802 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.039854 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.040085 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.040145 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:29:08.040418 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.030691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44539 (local address 127.13.15.129:52319)
0929 06:29:08.030850 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:08.030891 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:08.032222 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:08.032461 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:08.032466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:08.032766 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:08.032945 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.032949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.033924 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.033925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.034828 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.034830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.035374 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.035375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.035469 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.035976 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:08.035984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:08.037340 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:08.039163 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.039169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.039170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.039370 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.039538 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.039539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.039539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.039629 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.039860 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:08.039865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:08.039974 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 06:29:08.040057 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 06:29:08.040178 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260929 06:29:08.041146 19136 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44539
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
I20260929 06:29:08.041252 19136 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:08.041437 19136 heartbeater.cc:511] Master 127.13.15.190:44539 requested a full tablet report, sending...
I20260929 06:29:08.041977 18945 ts_manager.cc:194] Registered new tserver with Master: ca2e90f5e4dc4f638ece286fba4021d8 (127.13.15.129:33991)
I20260929 06:29:08.042605 18945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:52319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:29:08.056303 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:08.135753 19141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:08.135946 19141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:08.135967 19141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:08.137010 19141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:08.137041 19141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:08.137066 19141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:08.138448 19141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:08.138618 19141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:08.138775 19141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:08.143414 19149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:08.143476 19147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:08.143422 19146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:08.143461 19141 server_base.cc:1034] running on GCE node
I20260929 06:29:08.143728 19141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:08.143908 19141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:08.145041 19141 hybrid_clock.cc:648] HybridClock initialized: now 1790663348145014 us; error 47 us; skew 500 ppm
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.147374 19141 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:29:08.147784 19141 webserver.cc:460] Webserver started at http://127.13.15.130:41965/ using document root <none> and password file <none>
I20260929 06:29:08.148005 19141 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:08.148077 19141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:08.148219 19141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:08.149217 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "0f2202121fb04bd1a6b300bbb52a79ed"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "bfeed956e613726937ad1dcd45e5fa5e"
server_key_iv: "20ca9b70f1bea80795750692a579d0d1"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.149572 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "0f2202121fb04bd1a6b300bbb52a79ed"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "bfeed956e613726937ad1dcd45e5fa5e"
server_key_iv: "20ca9b70f1bea80795750692a579d0d1"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.151028 19141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:08.151728 19156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.151937 19141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:08.152029 19141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "0f2202121fb04bd1a6b300bbb52a79ed"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "bfeed956e613726937ad1dcd45e5fa5e"
server_key_iv: "20ca9b70f1bea80795750692a579d0d1"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.152103 19141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:08.165645 19141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:08.167899 19141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:08.168057 19141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:08.168285 19141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:08.168591 19141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:08.168622 19141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.168661 19141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:08.168707 19141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.174310 19141 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43735
I20260929 06:29:08.174364 19269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43735 every 8 connection(s)
I20260929 06:29:08.174664 19141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:08.180971 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19141
I20260929 06:29:08.181087 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:08.181303 13374 external_mini_cluster.cc:1428] Setting key 95c4f37ccc3958431d8737e76fcfd074
I20260929 06:29:08.183769 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.175391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:47839 (local address 127.13.15.190:44539)
0929 06:29:08.175431 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.175433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.175619 (+ 186us) server_negotiation.cc:407] Connection header received
0929 06:29:08.176471 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.176472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.176480 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.176533 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.177069 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.177528 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.178596 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.178774 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.178826 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.180750 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.180752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.180752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.180758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.182393 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.182825 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.182826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.182827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.182883 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.183148 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.183149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.183150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.183263 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.183351 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.183598 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.183685 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:29:08.183938 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.175349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44539 (local address 127.13.15.130:47839)
0929 06:29:08.175494 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:08.175532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:08.176392 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:08.176603 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:08.176607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:08.176840 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:08.177004 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.177008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.177624 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.177625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.178530 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.178532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.178841 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.178842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.178931 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.179415 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:08.179423 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:08.180672 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:08.182497 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.182503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.182505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.182737 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.182956 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.182956 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.182957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.183065 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.183340 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:08.183346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:08.183475 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 06:29:08.183579 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 06:29:08.183718 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260929 06:29:08.184376 19270 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44539
I20260929 06:29:08.184470 19270 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:08.184629 19270 heartbeater.cc:511] Master 127.13.15.190:44539 requested a full tablet report, sending...
I20260929 06:29:08.185010 18945 ts_manager.cc:194] Registered new tserver with Master: 0f2202121fb04bd1a6b300bbb52a79ed (127.13.15.130:43735)
I20260929 06:29:08.185391 18945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:47839
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:29:08.204963 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:08.285382 19275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:08.285590 19275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:08.285614 19275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:08.286753 19275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:08.286790 19275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:08.286816 19275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:08.288183 19275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:08.288352 19275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:08.288512 19275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:29:08.293437 19275 server_base.cc:1034] running on GCE node
W20260929 06:29:08.293396 19281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:08.293385 19283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:08.293402 19280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:08.293848 19275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:08.294083 19275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:08.295266 19275 hybrid_clock.cc:648] HybridClock initialized: now 1790663348295254 us; error 30 us; skew 500 ppm
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.297605 19275 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:29:08.298012 19275 webserver.cc:460] Webserver started at http://127.13.15.131:32779/ using document root <none> and password file <none>
I20260929 06:29:08.298305 19275 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:08.298370 19275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:08.298513 19275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:08.299520 19275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "e20e7b4fac6f4e83b38ab631f940db45"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "d6a5738599663271c75b1abfe4959146"
server_key_iv: "b48b22b24e81250e94e240a4405462d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.299881 19275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "e20e7b4fac6f4e83b38ab631f940db45"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "d6a5738599663271c75b1abfe4959146"
server_key_iv: "b48b22b24e81250e94e240a4405462d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.301118 19275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:08.301781 19290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.301944 19275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:08.302033 19275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "e20e7b4fac6f4e83b38ab631f940db45"
format_stamp: "Formatted at 2026-09-29 06:29:08 on dist-test-slave-sm0p"
server_key: "d6a5738599663271c75b1abfe4959146"
server_key_iv: "b48b22b24e81250e94e240a4405462d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:08.302119 19275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:08.310913 19275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:08.312518 19275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:08.312665 19275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:08.312886 19275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:08.313172 19275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:08.313225 19275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.313273 19275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:08.313301 19275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:08.318899 19275 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:44845
I20260929 06:29:08.318967 19403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:44845 every 8 connection(s)
I20260929 06:29:08.319267 19275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:08.319653 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19275
I20260929 06:29:08.319748 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:08.319924 13374 external_mini_cluster.cc:1428] Setting key fc8f59afb34c185bed713095cebfbb6c
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:08.328192 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.320267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:56013 (local address 127.13.15.190:44539)
0929 06:29:08.320378 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.320380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.320402 (+ 22us) server_negotiation.cc:407] Connection header received
0929 06:29:08.321452 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.321453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.321461 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.321507 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.322126 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.322796 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.323682 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.323851 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.323898 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.325862 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.325863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.325863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.325869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.327050 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.327399 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.327400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.327401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.327469 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.327693 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.327694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.327694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.327800 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.327856 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.328064 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.328131 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 06:29:08.328341 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.320013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44539 (local address 127.13.15.131:56013)
0929 06:29:08.320207 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:08.320253 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:08.321362 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:08.321577 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:08.321580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:08.321805 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:08.322051 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.322056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.322855 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.322855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.323615 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:08.323617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.323915 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.323915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:08.324003 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.324482 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:08.324497 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:08.325786 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:08.327113 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.327118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.327119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.327311 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.327537 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:08.327537 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:08.327537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:08.327631 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:08.327855 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:08.327858 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:08.327954 (+ 96us) client_negotiation.cc:769] Sending connection context
0929 06:29:08.328029 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 06:29:08.328133 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260929 06:29:08.328809 19404 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44539
I20260929 06:29:08.328881 19404 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:08.329008 19404 heartbeater.cc:511] Master 127.13.15.190:44539 requested a full tablet report, sending...
I20260929 06:29:08.329317 18945 ts_manager.cc:194] Registered new tserver with Master: e20e7b4fac6f4e83b38ab631f940db45 (127.13.15.131:44845)
I20260929 06:29:08.329670 18945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:56013
I20260929 06:29:08.331688 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/test-admin.keytab.
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.345881 13374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 06:29:08.347556 19411 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:08.346723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.13.15.129:33991)
0929 06:29:08.346899 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.346904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.346913 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:08.346944 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.346945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.346958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.347030 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.347474 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43176: BlockingRecv error: recv got EOF from 127.0.0.1:43176 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
W20260929 06:29:08.347707 13374 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc)
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.348680 13374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.349584 13374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:29:08.357292 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.350084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.13.15.129:33991)
0929 06:29:08.350126 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.350127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.350205 (+ 78us) server_negotiation.cc:407] Connection header received
0929 06:29:08.350273 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.350273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.350282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.350330 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.350693 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.351479 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.352252 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.352601 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.352655 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.354699 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.354705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.354706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.354712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.356294 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.356587 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.356590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.356590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.356642 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.356796 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.356796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.356797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.356909 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.357007 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.357063 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.357127 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 06:29:08.359057 19411 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:08.358521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.13.15.129:33991)
0929 06:29:08.358579 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.358581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.358624 (+ 43us) server_negotiation.cc:407] Connection header received
0929 06:29:08.358693 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.358694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.358703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.358757 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.359011 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43196: BlockingRecv error: recv got EOF from 127.0.0.1:43196 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 06:29:08.359143 13374 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc' of type FILE
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.359987 13374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.360770 13374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:29:08.366806 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.361233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.13.15.129:33991)
0929 06:29:08.361264 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.361267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.361293 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:29:08.361337 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.361337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.361344 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.361384 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.361678 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.362097 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.362721 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.362894 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.362969 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.364672 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.364673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.364673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.364679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.365963 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.366307 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.366307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.366308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.366352 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.366489 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.366490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.366490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.366559 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.366626 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.366703 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.366753 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 06:29:08.368089 19411 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:08.367658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.13.15.129:33991)
0929 06:29:08.367691 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.367692 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.367724 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:29:08.367777 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.367777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.367783 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.367822 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.368059 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 06:29:08.368149 13374 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790663323716959-13374-0/krb5kdc/krb5cc' of type FILE
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.368978 13374 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:08.369760 13374 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[18900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:29:08.375571 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:08.370228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.13.15.129:33991)
0929 06:29:08.370272 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:08.370274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:08.370303 (+ 29us) server_negotiation.cc:407] Connection header received
0929 06:29:08.370345 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:08.370346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:08.370352 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:08.370396 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:08.370693 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.371100 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.371679 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:08.371835 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:08.371877 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:08.373637 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:08.373638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:08.373638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:08.373643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:08.374808 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.375079 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.375079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.375080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.375125 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:08.375261 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:08.375262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:08.375262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:08.375332 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:08.375390 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:08.375470 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 06:29:08.375518 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 06:29:08.376216 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19007
I20260929 06:29:08.380975 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19141
I20260929 06:29:08.385043 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19275
I20260929 06:29:08.389089 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 18913
2026-09-29T06:29:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1802 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[19424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[19424](info): set up 2 sockets
Sep 29 06:29:08 dist-test-slave-sm0p krb5kdc[19424](info): commencing operation
krb5kdc: starting...
W20260929 06:29:09.402262 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab.
Entry for principal oryx/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab.
I20260929 06:29:09.512354 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38561
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab
--principal=oryx/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:38561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:09.590932 19437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:09.591133 19437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:09.591163 19437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:09.592221 19437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:09.592253 19437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:09.592267 19437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:09.592280 19437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:09.592293 19437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:09.593632 19437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:38561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab
--principal=oryx/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:09.593804 19437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:09.593956 19437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:09.598660 19445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:09.598670 19442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:09.598747 19437 server_base.cc:1034] running on GCE node
W20260929 06:29:09.598693 19443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:09.599138 19437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:09.599454 19437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:09.600605 19437 hybrid_clock.cc:648] HybridClock initialized: now 1790663349600595 us; error 26 us; skew 500 ppm
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:09.602826 19437 init.cc:377] Logged in from keytab as oryx/127.13.15.190@KRBTEST.COM (short username oryx)
I20260929 06:29:09.603175 19437 webserver.cc:460] Webserver started at http://127.13.15.190:37349/ using document root <none> and password file <none>
I20260929 06:29:09.603338 19437 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:09.603376 19437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:09.603462 19437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:09.604540 19437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "484ecb53ae9748e4b8848210efd2a507"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "2d57b4bc8b9b4c300beea52ee310defa"
server_key_iv: "7c877c0218870d24184f4cf4ce70900a"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.604848 19437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "484ecb53ae9748e4b8848210efd2a507"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "2d57b4bc8b9b4c300beea52ee310defa"
server_key_iv: "7c877c0218870d24184f4cf4ce70900a"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.606204 19437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:09.606904 19452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.607050 19437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:09.607151 19437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "484ecb53ae9748e4b8848210efd2a507"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "2d57b4bc8b9b4c300beea52ee310defa"
server_key_iv: "7c877c0218870d24184f4cf4ce70900a"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.607203 19437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:09.616667 19437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:09.618173 19437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:09.618311 19437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:09.621763 19437 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:38561
I20260929 06:29:09.621814 19504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:38561 every 8 connection(s)
I20260929 06:29:09.622115 19437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:09.623071 19505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:09.625136 19505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Bootstrap starting.
I20260929 06:29:09.625686 19505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:09.625967 19505 log.cc:824] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:09.626564 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19437
I20260929 06:29:09.626682 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:09.626748 19505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: No bootstrap required, opened a new log
I20260929 06:29:09.626875 13374 external_mini_cluster.cc:1428] Setting key 077d9e96a1b1661a21c48f04c93af4d0
I20260929 06:29:09.629040 19505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } }
I20260929 06:29:09.629163 19505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:09.629184 19505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 484ecb53ae9748e4b8848210efd2a507, State: Initialized, Role: FOLLOWER
I20260929 06:29:09.629300 19505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } }
I20260929 06:29:09.629369 19505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:09.629410 19505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:09.629454 19505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:09.629953 19505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } }
I20260929 06:29:09.630030 19505 leader_election.cc:302] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 484ecb53ae9748e4b8848210efd2a507; no voters:
I20260929 06:29:09.630285 19505 leader_election.cc:288] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:09.630303 19510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:09.630496 19510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 LEADER]: Becoming Leader. State: Replica: 484ecb53ae9748e4b8848210efd2a507, State: Running, Role: LEADER
I20260929 06:29:09.630575 19510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } }
I20260929 06:29:09.630610 19505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:09.630869 19512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484ecb53ae9748e4b8848210efd2a507. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } }
I20260929 06:29:09.630882 19511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } }
I20260929 06:29:09.631098 19511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:09.631021 19512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:09.631726 19514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:09.632192 19514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:09.637389 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:09.627340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.13.15.190:38561)
0929 06:29:09.627518 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:09.627527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:09.627565 (+ 38us) server_negotiation.cc:407] Connection header received
0929 06:29:09.628060 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:09.628066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:09.628107 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:09.628210 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:09.628933 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.630225 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.631308 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.631895 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.632009 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:09.634341 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:09.634345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:09.634346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:09.634356 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:09.635845 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.636201 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.636203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.636204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.636294 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.636510 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.636511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.636511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.636736 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:09.636927 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:09.637000 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 06:29:09.637144 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260929 06:29:09.639055 19514 catalog_manager.cc:1347] Generated new cluster ID: 908d4c18b8ec42ecaa9a31f858c56dfc
I20260929 06:29:09.639102 19514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:09.654310 19514 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:09.654817 19514 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:09.664750 19514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Generated new TSK 0
I20260929 06:29:09.664950 19514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
I20260929 06:29:09.698225 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:09.780623 19531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:09.780810 19531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:09.780831 19531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:09.781845 19531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:09.781875 19531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:09.781900 19531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:09.783345 19531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:09.783499 19531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:09.783653 19531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:09.788403 19539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:09.788427 19537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:09.788686 19531 server_base.cc:1034] running on GCE node
W20260929 06:29:09.788427 19536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:09.788995 19531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:09.789250 19531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:09.790405 19531 hybrid_clock.cc:648] HybridClock initialized: now 1790663349790383 us; error 33 us; skew 500 ppm
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:09.792761 19531 init.cc:377] Logged in from keytab as oryx/127.13.15.129@KRBTEST.COM (short username oryx)
I20260929 06:29:09.793133 19531 webserver.cc:460] Webserver started at http://127.13.15.129:41201/ using document root <none> and password file <none>
I20260929 06:29:09.793300 19531 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:09.793339 19531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:09.793421 19531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:09.794420 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "e715f40d0ed9491b963edc65cbc261af"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "c5b161d13424aa9025c774af0ec26ca0"
server_key_iv: "08376661a753c362706c310b3a42c4ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.794713 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "e715f40d0ed9491b963edc65cbc261af"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "c5b161d13424aa9025c774af0ec26ca0"
server_key_iv: "08376661a753c362706c310b3a42c4ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.795957 19531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:09.796581 19546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.796736 19531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:09.796815 19531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "e715f40d0ed9491b963edc65cbc261af"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "c5b161d13424aa9025c774af0ec26ca0"
server_key_iv: "08376661a753c362706c310b3a42c4ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.796885 19531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:09.812736 19531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:09.815114 19531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:09.815277 19531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:09.815511 19531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:09.815831 19531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:09.815863 19531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.815886 19531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:09.815933 19531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.821337 19531 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:37749
I20260929 06:29:09.821403 19659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:37749 every 8 connection(s)
I20260929 06:29:09.821754 19531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:09.823115 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19531
I20260929 06:29:09.823211 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:09.823418 13374 external_mini_cluster.cc:1428] Setting key ef9b4bfb1e0e80ba0fed5e8524e8468a
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:29:09.832028 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:09.822510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53357 (local address 127.13.15.190:38561)
0929 06:29:09.822568 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:09.822570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:09.822795 (+ 225us) server_negotiation.cc:407] Connection header received
0929 06:29:09.823593 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:09.823595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:09.823620 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:09.823692 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:09.825075 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.825560 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.826672 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.826881 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.826968 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:09.829000 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:09.829002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:09.829002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:09.829010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:09.830517 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.831003 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.831004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.831005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.831066 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.831349 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.831350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.831350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.831460 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:09.831537 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:09.831815 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 06:29:09.831918 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:29:09.832259 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:09.822483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:53357)
0929 06:29:09.822668 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:09.822702 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:09.823433 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:09.823865 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:09.823869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:09.824698 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:09.824980 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:09.824986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.825672 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.825673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:09.826573 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:09.826575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.826984 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.826986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:09.827110 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:09.827608 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:09.827616 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:09.828905 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:09.830618 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:09.830625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:09.830627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:09.830891 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:09.831142 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:09.831143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:09.831143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:09.831269 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:09.831526 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:09.831532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:09.831686 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 06:29:09.831794 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 06:29:09.831949 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260929 06:29:09.832716 19660 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:09.832825 19660 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:09.833019 19660 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:09.833562 19469 ts_manager.cc:194] Registered new tserver with Master: e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.130@KRBTEST.COM" created.
I20260929 06:29:09.834421 19469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
I20260929 06:29:09.848522 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:09.927525 19665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:09.927734 19665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:09.927759 19665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:09.928891 19665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:09.928972 19665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:09.929005 19665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:09.930501 19665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:09.930702 19665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:09.930872 19665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:09.935881 19673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:09.935927 19671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:09.935882 19670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:09.936045 19665 server_base.cc:1034] running on GCE node
I20260929 06:29:09.936281 19665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:09.936528 19665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:09.937671 19665 hybrid_clock.cc:648] HybridClock initialized: now 1790663349937652 us; error 33 us; skew 500 ppm
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:09.940075 19665 init.cc:377] Logged in from keytab as oryx/127.13.15.130@KRBTEST.COM (short username oryx)
I20260929 06:29:09.940477 19665 webserver.cc:460] Webserver started at http://127.13.15.130:42741/ using document root <none> and password file <none>
I20260929 06:29:09.940706 19665 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:09.940770 19665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:09.940914 19665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:09.941923 19665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "290a56d518e84239aa611bfda9b693ee"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "df1be8a6c8e67d9909e0e1cb6d1d684e"
server_key_iv: "58cb4fcdefdd5ee92caa4c984c5e2a94"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.942310 19665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "290a56d518e84239aa611bfda9b693ee"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "df1be8a6c8e67d9909e0e1cb6d1d684e"
server_key_iv: "58cb4fcdefdd5ee92caa4c984c5e2a94"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.943655 19665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:09.944471 19680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.944674 19665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:09.944770 19665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "290a56d518e84239aa611bfda9b693ee"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "df1be8a6c8e67d9909e0e1cb6d1d684e"
server_key_iv: "58cb4fcdefdd5ee92caa4c984c5e2a94"
server_key_version: "encryptionkey@0"
I20260929 06:29:09.944850 19665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:09.955545 19665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:09.957263 19665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:09.957420 19665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:09.957633 19665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:09.957950 19665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:09.958005 19665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.958040 19665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:09.958068 19665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:09.963960 19665 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43017
I20260929 06:29:09.964010 19793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43017 every 8 connection(s)
I20260929 06:29:09.964314 19665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:29:09 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:09.973114 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19665
I20260929 06:29:09.973243 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:09.973404 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:09.964960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:49661 (local address 127.13.15.190:38561)
0929 06:29:09.965040 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:09.965043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:09.965225 (+ 182us) server_negotiation.cc:407] Connection header received
0929 06:29:09.965759 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:09.965760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:09.965774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:09.965827 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:09.966801 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.967298 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.968241 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.968443 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.968517 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:09.970512 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:09.970514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:09.970514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:09.970521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:09.971979 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.972424 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.972425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.972426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.972483 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:09.972757 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:09.972758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:09.972759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:09.972874 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:09.972945 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:09.973236 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 06:29:09.973316 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:29:09.973459 13374 external_mini_cluster.cc:1428] Setting key f531c28ce2cc57b323cacbe147374264
I20260929 06:29:09.973583 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:09.964933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:49661)
0929 06:29:09.965111 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:09.965145 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:09.965632 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:09.965901 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:09.965904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:09.966515 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:09.966701 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:09.966705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.967389 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.967390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:09.968145 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:09.968147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:09.968520 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:09.968521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:09.968649 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:09.969153 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:09.969161 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:09.970417 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:09.972092 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:09.972098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:09.972107 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:09.972338 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:09.972571 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:09.972572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:09.972573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:09.972689 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:09.972969 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:09.972975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:09.973106 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 06:29:09.973206 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 06:29:09.973353 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260929 06:29:09.974267 19794 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:09.974355 19794 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:09.974541 19794 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:09.974931 19468 ts_manager.cc:194] Registered new tserver with Master: 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:09.975306 19468 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:49661
WARNING: no policy specified for oryx/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
I20260929 06:29:09.996975 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:10.075778 19799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:10.075975 19799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:10.075999 19799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:10.077055 19799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:10.077087 19799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:10.077126 19799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:10.078480 19799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:10.078661 19799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:10.078823 19799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.083729 19804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.083829 19799 server_base.cc:1034] running on GCE node
W20260929 06:29:10.083789 19807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.083750 19805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.084147 19799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.084386 19799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.085551 19799 hybrid_clock.cc:648] HybridClock initialized: now 1790663350085537 us; error 52 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.088073 19799 init.cc:377] Logged in from keytab as oryx/127.13.15.131@KRBTEST.COM (short username oryx)
I20260929 06:29:10.088485 19799 webserver.cc:460] Webserver started at http://127.13.15.131:34605/ using document root <none> and password file <none>
I20260929 06:29:10.088702 19799 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.088768 19799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.088914 19799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:10.089931 19799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "b2b8f98c7cc041769103428c09d47a43"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "1a6624ce610579a1bb6df5506b0759f5"
server_key_iv: "650828fd4ca2676fd788d622a10469c4"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.090313 19799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "b2b8f98c7cc041769103428c09d47a43"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "1a6624ce610579a1bb6df5506b0759f5"
server_key_iv: "650828fd4ca2676fd788d622a10469c4"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.091585 19799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:10.092269 19814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.092437 19799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:10.092532 19799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "b2b8f98c7cc041769103428c09d47a43"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "1a6624ce610579a1bb6df5506b0759f5"
server_key_iv: "650828fd4ca2676fd788d622a10469c4"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.092610 19799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.122706 19799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.125121 19799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:10.125301 19799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.125548 19799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:10.125936 19799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:10.125991 19799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.126026 19799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:10.126073 19799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.132083 19799 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:44441
I20260929 06:29:10.132122 19927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:44441 every 8 connection(s)
I20260929 06:29:10.132469 19799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:10.141206 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.133171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:55207 (local address 127.13.15.190:38561)
0929 06:29:10.133233 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.133235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.133431 (+ 196us) server_negotiation.cc:407] Connection header received
0929 06:29:10.133880 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.133881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.133890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.133939 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.134963 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.135450 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.136370 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.136559 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.136602 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.138710 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.138711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.138712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.138718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.140123 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.140458 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.140471 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.140472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.140516 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.140725 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.140725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.140725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.140816 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.140863 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.141077 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.141145 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:29:10.141386 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.133178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:55207)
0929 06:29:10.133308 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.133341 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.133801 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.133990 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.133993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.134601 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.134865 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.134871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.135524 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.135525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.136301 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.136302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.136620 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.136620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.136714 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.137136 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.137144 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.138627 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.140185 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.140189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.140190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.140398 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.140568 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.140569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.140569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.140660 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.140866 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.140870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.140974 (+ 104us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.141052 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.141160 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":76,"threads_started":1}
I20260929 06:29:10.141947 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19799
I20260929 06:29:10.141973 19928 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:10.142037 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:10.142071 19928 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.142239 13374 external_mini_cluster.cc:1428] Setting key 304c0ee44b2f538b9147df7a412d73df
I20260929 06:29:10.142262 19928 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:10.142596 19469 ts_manager.cc:194] Registered new tserver with Master: b2b8f98c7cc041769103428c09d47a43 (127.13.15.131:44441)
I20260929 06:29:10.143046 19469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:55207
I20260929 06:29:10.143114 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.189.keytab.
Entry for principal oryx/127.13.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.189.keytab.
I20260929 06:29:10.166921 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37085
--webserver_interface=127.13.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.189.keytab
--principal=oryx/127.13.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:38561,127.13.15.189:37085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:10.245671 19933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:10.245882 19933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:10.245903 19933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:10.246975 19933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:10.247008 19933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:10.247022 19933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:10.247035 19933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:10.247046 19933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:10.248430 19933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:38561,127.13.15.189:37085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.189.keytab
--principal=oryx/127.13.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
W20260929 06:29:10.248521 19933 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.15.190:38561,127.13.15.189:37085'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 06:29:10.248622 19933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:10.248781 19933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.253444 19939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.253412 19941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.253413 19938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.253727 19933 server_base.cc:1034] running on GCE node
I20260929 06:29:10.253985 19933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.254307 19933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.255476 19933 hybrid_clock.cc:648] HybridClock initialized: now 1790663350255465 us; error 27 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.257925 19933 init.cc:377] Logged in from keytab as oryx/127.13.15.189@KRBTEST.COM (short username oryx)
I20260929 06:29:10.258384 19933 webserver.cc:460] Webserver started at http://127.13.15.189:35559/ using document root <none> and password file <none>
I20260929 06:29:10.258558 19933 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.258596 19933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.258683 19933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:10.259636 19933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/instance:
uuid: "23abb0979d814a20bfbe50e0ebd31cce"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "c9d2b5543336ca3ce29f9f4263b072ee"
server_key_iv: "37de2ee780d8f55b5359cf731556a587"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.259918 19933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal/instance:
uuid: "23abb0979d814a20bfbe50e0ebd31cce"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "c9d2b5543336ca3ce29f9f4263b072ee"
server_key_iv: "37de2ee780d8f55b5359cf731556a587"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.261240 19933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:10.261865 19948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.262037 19933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:10.262116 19933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "23abb0979d814a20bfbe50e0ebd31cce"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "c9d2b5543336ca3ce29f9f4263b072ee"
server_key_iv: "37de2ee780d8f55b5359cf731556a587"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.262221 19933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.279400 19933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.280978 19933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:10.281071 19933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.284657 19933 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.189:37085
I20260929 06:29:10.284680 20000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.189:37085 every 8 connection(s)
I20260929 06:29:10.285032 19933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
I20260929 06:29:10.285971 20001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:10.287447 20001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:29:10.291162 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 19933
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.189@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:10.296568 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.288083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.13.15.190:38561)
0929 06:29:10.288149 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.288150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.288346 (+ 196us) server_negotiation.cc:407] Connection header received
0929 06:29:10.288810 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.288812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.288823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.288883 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.289441 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.289918 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.290919 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.291219 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.291295 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.293350 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.293351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.293352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.293358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.294799 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.295542 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.295543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.295543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.295594 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.295888 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.295889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.295889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.296015 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.296065 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.296401 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.296459 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:29:10.296753 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.288071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52110)
0929 06:29:10.288232 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.288263 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.288709 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.288951 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.288954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.289175 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.289345 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.289349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.290005 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.290007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.290828 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.290831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.291311 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.291313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.291431 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.291842 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.291849 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.293265 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.294873 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.294877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.294879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.295164 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.295689 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.295690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.295690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.295819 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.296081 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.296086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.296260 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.296378 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.296535 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
I20260929 06:29:10.297273 20001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:29:10.300449 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.291634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.13.15.189:37085)
0929 06:29:10.291765 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.291768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.291913 (+ 145us) server_negotiation.cc:407] Connection header received
0929 06:29:10.291982 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.291985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.292005 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.292080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.292790 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.293565 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.294457 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.294905 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.294959 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.297720 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.297726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.297726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.297732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.299229 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.299547 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.299550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.299550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.299602 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.299785 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.299786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.299786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.299924 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.300089 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.300154 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.300227 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.189@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
I20260929 06:29:10.302935 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.297487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33688)
0929 06:29:10.297536 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.297545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.297585 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.297776 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.297776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.297937 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.297980 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.297982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.298626 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.298627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.299157 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.299159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.299493 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.299494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.299549 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.299907 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.299911 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.301092 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.302260 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.302261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.302262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.302397 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.302529 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.302529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.302529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.302564 (+ 35us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.302747 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.302747 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.302776 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.302822 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.302884 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 06:29:10.303051 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.297523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.13.15.189:37085)
0929 06:29:10.297657 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.297659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.297666 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:10.297683 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.297683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.297691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.297775 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.298074 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.298543 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.299236 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.299422 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.299492 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.301159 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.301159 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.301160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.301164 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.302124 (+ 960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.302450 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.302450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.302451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.302494 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.302594 (+ 100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.302594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.302594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.302675 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.302737 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.302835 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.302924 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260929 06:29:10.304092 20001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Bootstrap starting.
I20260929 06:29:10.304669 20001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:10.304927 20001 log.cc:824] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:10.305572 20001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: No bootstrap required, opened a new log
I20260929 06:29:10.306957 20001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } }
I20260929 06:29:10.307071 20001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:10.307092 20001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23abb0979d814a20bfbe50e0ebd31cce, State: Initialized, Role: FOLLOWER
I20260929 06:29:10.307245 20001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } }
I20260929 06:29:10.307462 20008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } } }
I20260929 06:29:10.307497 20001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:10.307556 20008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:29:10.309020 20019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:29:10.309105 20019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:29:10.310561 19933 master_runner.cc:428] Detected that this master 23abb0979d814a20bfbe50e0ebd31cce is joining an existing cluster
I20260929 06:29:10.310621 19933 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.15.189:37085
I20260929 06:29:10.317231 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.311384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.13.15.190:38561)
0929 06:29:10.311419 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.311421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.311548 (+ 127us) server_negotiation.cc:407] Connection header received
0929 06:29:10.311613 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.311614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.311623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.311674 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.312020 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.312452 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.313503 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.313673 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.313719 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.314686 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.314687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.314687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.314694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.316121 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.316467 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.316468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.316468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.316523 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.316729 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.316730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.316731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.316830 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.316903 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.317063 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.317133 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:29:10.317653 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.311349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52112)
0929 06:29:10.311495 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.311504 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.311558 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.311681 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.311681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.311872 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.311951 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.311953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.312527 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.312528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.313425 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.313428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.313766 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.313767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.313827 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.314229 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.314234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.314593 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.316217 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.316217 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.316218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.316383 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.316602 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.316602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.316603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.316658 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.316895 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.316896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.316945 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.317025 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.317530 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260929 06:29:10.321183 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.311492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33694)
0929 06:29:10.314222 (+ 2730us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.314232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.314309 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.314459 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.314460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.314670 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.314733 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.314734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.315628 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.315629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.316456 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.316459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.316947 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.316947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.316999 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.317357 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.317361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.318187 (+ 826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.320108 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.320109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.320109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.320295 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.320474 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.320475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.320476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.320527 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.320857 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.320858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.320891 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.320968 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.321062 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2715,"thread_start_us":76,"threads_started":1}
I20260929 06:29:10.321193 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.311454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.13.15.189:37085)
0929 06:29:10.311483 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.311485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.314297 (+ 2812us) server_negotiation.cc:407] Connection header received
0929 06:29:10.314367 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.314367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.314379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.314443 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.314814 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.315540 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.316559 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.316848 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.316886 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.318253 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.318254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.318254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.318260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.320027 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.320360 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.320361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.320361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.320405 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.320705 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.320706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.320706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.320788 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.320854 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.320987 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.321111 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":14}
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.190@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
I20260929 06:29:10.327764 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.320073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33710 (local address 127.13.15.189:37085)
0929 06:29:10.320106 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.320108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.320498 (+ 390us) server_negotiation.cc:407] Connection header received
0929 06:29:10.320618 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.320619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.320627 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.320677 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.321203 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.321896 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.323021 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.323202 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.323249 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.325115 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.325116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.325117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.325125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.326586 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.326999 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.327000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.327000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.327054 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.327265 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.327266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.327266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.327363 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.327434 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.327605 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.327679 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:29:10.327792 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.320051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33710)
0929 06:29:10.320406 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.320449 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.320529 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.320685 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.320687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.321002 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.321062 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.321064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.321990 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.321991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.322937 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.322939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.323272 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.323273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.323347 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.323713 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.323717 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.325023 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.326707 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.326713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.326715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.326911 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.327136 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.327137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.327137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.327194 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.327429 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.327436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.327483 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.327580 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.327694 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":71,"threads_started":1}
I20260929 06:29:10.328321 19469 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.15.189:37085: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce"
member_type: NON_VOTER
last_known_addr {
host: "127.13.15.189"
port: 37085
}
attrs {
promote: true
}
}
dest_uuid: "484ecb53ae9748e4b8848210efd2a507"
cas_config_opid_index: -1
I20260929 06:29:10.328619 19469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } }
I20260929 06:29:10.329099 20029 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 23abb0979d814a20bfbe50e0ebd31cce (127.13.15.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } }
I20260929 06:29:10.329442 19975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:10.330019 20029 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.15.189:37085
I20260929 06:29:10.330232 19933 master.cc:537] Master@127.13.15.189:37085 shutting down...
I20260929 06:29:10.330348 19975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Refusing update from remote peer 484ecb53ae9748e4b8848210efd2a507: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 06:29:10.330276 20028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } }
I20260929 06:29:10.330484 20028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:10.330667 20028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 06:29:10.331496 19456 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 06:29:10.331897 19456 consensus_peers.cc:487] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 -> Peer 23abb0979d814a20bfbe50e0ebd31cce (127.13.15.189:37085): Couldn't send request to peer 23abb0979d814a20bfbe50e0ebd31cce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 06:29:10.333299 19933 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:10.333380 19933 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:10.333428 19933 tablet_replica.cc:330] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: stopping tablet replica
I20260929 06:29:10.345973 19933 master.cc:552] Master@127.13.15.189:37085 shutdown complete.
I20260929 06:29:10.346036 19933 master_runner.cc:305] Clearing existing system tablet
I20260929 06:29:10.346216 19933 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.346273 19933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.346843 19933 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:10.347353 20033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.347481 19933 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.347540 19933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.347569 19933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "23abb0979d814a20bfbe50e0ebd31cce"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "c9d2b5543336ca3ce29f9f4263b072ee"
server_key_iv: "37de2ee780d8f55b5359cf731556a587"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.348222 19933 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:10.349484 19933 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 06:29:10.349581 19933 log.cc:1196] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 06:29:10.349766 19933 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Deleting consensus metadata
I20260929 06:29:10.349898 19933 master_runner.cc:315] Copying system tablet from 127.13.15.190:38561
I20260929 06:29:10.350544 19933 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: tablet copy: Beginning tablet copy session from remote peer at address 127.13.15.190:38561
I20260929 06:29:10.356665 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.350826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52122)
0929 06:29:10.350916 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.350926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.350984 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.351124 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.351125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.351305 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.351371 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.351373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.352261 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.352262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.352760 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.352761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.353190 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.353191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.353235 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.353690 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.353695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.354011 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.355764 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.355765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.355765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.355920 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.356143 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.356143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.356144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.356191 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.356436 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.356437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.356470 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.356518 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.356590 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":56,"threads_started":1}
I20260929 06:29:10.356721 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.350883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.13.15.190:38561)
0929 06:29:10.350924 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.350926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.350982 (+ 56us) server_negotiation.cc:407] Connection header received
0929 06:29:10.351026 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.351027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.351038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.351092 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.351444 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.352185 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.352832 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.353121 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.353188 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.354109 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.354111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.354111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.354118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.355687 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.355992 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.355993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.355994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.356059 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.356250 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.356251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.356251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.356374 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.356443 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.356540 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.356617 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:29:10.357277 19489 tablet_copy_service.cc:140] P 484ecb53ae9748e4b8848210efd2a507: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 23abb0979d814a20bfbe50e0ebd31cce ({username='oryx', principal='oryx/127.13.15.189@KRBTEST.COM'} at 127.0.0.1:52122)
I20260929 06:29:10.357371 19489 tablet_copy_service.cc:161] P 484ecb53ae9748e4b8848210efd2a507: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 23abb0979d814a20bfbe50e0ebd31cce at {username='oryx', principal='oryx/127.13.15.189@KRBTEST.COM'} at 127.0.0.1:52122: session id = 23abb0979d814a20bfbe50e0ebd31cce-00000000000000000000000000000000
I20260929 06:29:10.358167 19489 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 06:29:10.358747 19933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 06:29:10.359594 19933 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: tablet copy: Starting download of 0 data blocks...
I20260929 06:29:10.359743 19933 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: tablet copy: Starting download of 1 WAL segments...
I20260929 06:29:10.360562 19933 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 06:29:10.361347 19489 tablet_copy_service.cc:342] P 484ecb53ae9748e4b8848210efd2a507: Request end of tablet copy session 23abb0979d814a20bfbe50e0ebd31cce-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.15.189@KRBTEST.COM'} at 127.0.0.1:52122
I20260929 06:29:10.361404 19489 tablet_copy_service.cc:433] P 484ecb53ae9748e4b8848210efd2a507: ending tablet copy session 23abb0979d814a20bfbe50e0ebd31cce-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 23abb0979d814a20bfbe50e0ebd31cce
I20260929 06:29:10.363504 19933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.364578 20043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.364616 20041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.364619 20040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.364877 19933 server_base.cc:1034] running on GCE node
I20260929 06:29:10.364977 19933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.365104 19933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.366184 19933 hybrid_clock.cc:648] HybridClock initialized: now 1790663350366193 us; error 39 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.367345 19933 init.cc:377] Logged in from keytab as oryx/127.13.15.189@KRBTEST.COM (short username oryx)
I20260929 06:29:10.367583 19933 webserver.cc:460] Webserver started at http://127.13.15.189:36471/ using document root <none> and password file <none>
I20260929 06:29:10.367707 19933 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.367759 19933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.368296 19933 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:10.368849 20050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.368966 19933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:10.369010 19933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "23abb0979d814a20bfbe50e0ebd31cce"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "c9d2b5543336ca3ce29f9f4263b072ee"
server_key_iv: "37de2ee780d8f55b5359cf731556a587"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.369063 19933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.388418 19933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.390004 19933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.393162 19933 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.189:37085
I20260929 06:29:10.393378 20102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.189:37085 every 8 connection(s)
I20260929 06:29:10.393540 19933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
I20260929 06:29:10.394271 20103 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:29:10.394666 20103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Bootstrap starting.
I20260929 06:29:10.396921 20103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:29:10.397141 20103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: Bootstrap complete.
I20260929 06:29:10.397352 20103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } }
I20260929 06:29:10.397415 20103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 23abb0979d814a20bfbe50e0ebd31cce, State: Initialized, Role: LEARNER
I20260929 06:29:10.397440 20103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } }
I20260929 06:29:10.397603 20103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:10.397610 20106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } }
I20260929 06:29:10.397732 20106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: This master's current role is: LEARNER
I20260929 06:29:10.399060 20117 catalog_manager.cc:1259] Loaded cluster ID: 908d4c18b8ec42ecaa9a31f858c56dfc
I20260929 06:29:10.399102 20117 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: loading cluster ID for follower catalog manager: success
I20260929 06:29:10.400206 20117 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: acquiring CA information for follower catalog manager: success
I20260929 06:29:10.400475 20117 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:29:10.407706 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.401662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33728 (local address 127.13.15.189:37085)
0929 06:29:10.401772 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.401774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.401781 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:10.401801 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.401802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.401811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.401886 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.402449 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.403037 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.403722 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.403930 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.404011 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.405137 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.405138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.405139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.405145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.406595 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.406945 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.406946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.406946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.407004 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.407203 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.407204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.407204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.407306 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.407410 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.407505 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.407586 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for oryx/127.13.15.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.188.keytab.
Entry for principal oryx/127.13.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.188.keytab.
I20260929 06:29:10.430583 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:42637
--webserver_interface=127.13.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.188.keytab
--principal=oryx/127.13.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:10.511308 20121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:10.511528 20121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:10.511552 20121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:10.512621 20121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:10.512658 20121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:10.512674 20121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:10.512688 20121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:10.512701 20121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:10.514079 20121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.188.keytab
--principal=oryx/127.13.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:42637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:10.514307 20121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:10.514518 20121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.519814 20127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.519825 20126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.520015 20121 server_base.cc:1034] running on GCE node
W20260929 06:29:10.519869 20129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.520298 20121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.520572 20121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.521730 20121 hybrid_clock.cc:648] HybridClock initialized: now 1790663350521711 us; error 32 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.524065 20121 init.cc:377] Logged in from keytab as oryx/127.13.15.188@KRBTEST.COM (short username oryx)
I20260929 06:29:10.524523 20121 webserver.cc:460] Webserver started at http://127.13.15.188:43133/ using document root <none> and password file <none>
I20260929 06:29:10.524791 20121 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.524858 20121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.525015 20121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:10.526050 20121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/instance:
uuid: "70e84e810d524072820ef06b20d79427"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "3e0f74cee3ec20bc7c70040333df30a4"
server_key_iv: "e72a3a0de77f24518cdc1cf6bf376565"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.526485 20121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal/instance:
uuid: "70e84e810d524072820ef06b20d79427"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "3e0f74cee3ec20bc7c70040333df30a4"
server_key_iv: "e72a3a0de77f24518cdc1cf6bf376565"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.527887 20121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:10.528550 20136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.528729 20121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:10.528823 20121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "70e84e810d524072820ef06b20d79427"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "3e0f74cee3ec20bc7c70040333df30a4"
server_key_iv: "e72a3a0de77f24518cdc1cf6bf376565"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.528903 20121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.550920 20121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.552672 20121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:10.552827 20121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.556761 20121 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.188:42637
I20260929 06:29:10.556795 20188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.188:42637 every 8 connection(s)
I20260929 06:29:10.557163 20121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
I20260929 06:29:10.558055 20189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:10.559505 20189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.188@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:10.565333 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20121
I20260929 06:29:10.567914 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.560255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.13.15.190:38561)
0929 06:29:10.560302 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.560304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.560394 (+ 90us) server_negotiation.cc:407] Connection header received
0929 06:29:10.560799 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.560800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.560807 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.560857 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.561392 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.561854 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.562784 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.562952 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.563002 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.564906 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.564908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.564908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.564914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.566591 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.566991 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.566991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.566992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.567040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.567290 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.567290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.567290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.567402 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.567451 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.567728 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.567784 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:29:10.568109 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.560125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52130)
0929 06:29:10.560276 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.560315 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.560726 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.560913 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.560916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.561146 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.561322 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.561325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.561925 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.561926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.562716 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.562718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.563027 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.563028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.563132 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.563553 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.563560 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.564820 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.566679 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.566683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.566684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.566906 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.567100 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.567100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.567101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.567228 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.567461 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.567466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.567601 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.567706 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.567883 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260929 06:29:10.568660 20189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.188@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
I20260929 06:29:10.574715 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.565915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.13.15.188:42637)
0929 06:29:10.566062 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.566067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.566088 (+ 21us) server_negotiation.cc:407] Connection header received
0929 06:29:10.566105 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.566106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.566123 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.566230 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.566916 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.567968 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.568904 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.569300 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.569380 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.571970 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.571978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.571978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.571983 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.573563 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.573915 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.573918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.573918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.573974 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.574187 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.574188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.574188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.574319 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.574466 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.574541 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.574608 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":98,"threads_started":1}
I20260929 06:29:10.576049 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.568907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33730)
0929 06:29:10.568940 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.568950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.568982 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.569152 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.569153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.569254 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.569294 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.569295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.570219 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.570220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.571014 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.571016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.571459 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.571460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.571524 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.571973 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.571978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.573404 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.575006 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.575007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.575007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.575213 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.575436 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.575437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.575438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.575498 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.575757 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.575758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.575798 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.575868 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.575957 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 06:29:10.576067 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.569007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.13.15.189:37085)
0929 06:29:10.569043 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.569045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.569052 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:10.569075 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.569076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.569087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.569141 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.569371 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.570083 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.571084 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.571381 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.571451 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.573486 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.573487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.573488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.573495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.574923 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.575295 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.575296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.575297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.575370 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.575567 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.575568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.575569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.575687 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.575764 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.575907 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.575987 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:29:10.576797 20189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 42637 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.188@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
I20260929 06:29:10.584335 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.577042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.0.0.1:52468)
0929 06:29:10.577103 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.577121 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.577163 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.577319 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.577320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.577457 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.577490 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.577492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.578451 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.578452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.579238 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.579240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.579771 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.579773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.579832 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.580258 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.580264 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.581776 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.583546 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.583547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.583548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.583753 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.583929 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.583930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.583930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.583977 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.584148 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.584149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.584177 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.584220 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.584284 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 06:29:10.584447 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.577073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.13.15.188:42637)
0929 06:29:10.577112 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.577114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.577150 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:29:10.577223 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.577224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.577233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.577294 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.577558 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.578357 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.579332 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.579668 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.579754 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.581876 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.581877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.581877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.581883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.583466 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.583829 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.583829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.583830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.583875 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.584025 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.584026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.584026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.584102 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.584143 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.584251 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.584325 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":22}
I20260929 06:29:10.585680 20189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Bootstrap starting.
I20260929 06:29:10.586535 20189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:10.586807 20189 log.cc:824] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:10.587441 20189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: No bootstrap required, opened a new log
I20260929 06:29:10.588775 20189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 42637 } }
I20260929 06:29:10.588891 20189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:10.588934 20189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e84e810d524072820ef06b20d79427, State: Initialized, Role: FOLLOWER
I20260929 06:29:10.589058 20189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 42637 } }
I20260929 06:29:10.589339 20189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:10.589310 20194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 42637 } } }
I20260929 06:29:10.589414 20194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:29:10.590924 20205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:29:10.590977 20205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:29:10.593220 20121 master_runner.cc:428] Detected that this master 70e84e810d524072820ef06b20d79427 is joining an existing cluster
I20260929 06:29:10.593266 20121 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.15.188:42637
I20260929 06:29:10.601035 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.0.0.1:52484)
0929 06:29:10.594598 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.594607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.594665 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.594767 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.594768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.594988 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.595066 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.595067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.596031 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.596032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.596896 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.596898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.597395 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.597396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.597464 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.597825 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.597830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.598303 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.599998 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.599999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.599999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.600195 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.600392 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.600393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.600439 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.600743 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.600744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.600769 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.600837 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.600926 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":78,"threads_started":1}
I20260929 06:29:10.601089 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.13.15.190:38561)
0929 06:29:10.594177 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.594179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.594563 (+ 384us) server_negotiation.cc:407] Connection header received
0929 06:29:10.594620 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.594621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.594631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.594675 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.595027 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.595562 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.596426 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.596646 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.596722 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.597832 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.597833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.597834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.597839 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.599722 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600106 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.600107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.600107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.600164 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600387 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.600388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.600388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.600518 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.600573 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.600911 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.601001 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:29:10.601043 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.13.15.188:42637)
0929 06:29:10.594416 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.594418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.594651 (+ 233us) server_negotiation.cc:407] Connection header received
0929 06:29:10.594709 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.594709 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.594716 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.594761 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.595149 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.595922 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.596996 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.597300 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.597335 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.598398 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.598399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.598399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.598405 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.599906 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600286 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.600287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.600287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.600331 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600501 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.600502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.600502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.600688 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.600751 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.600867 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.600970 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":14}
I20260929 06:29:10.601588 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52138)
0929 06:29:10.594492 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.594500 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.594563 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.594712 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.594713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.594907 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.594953 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.594955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.595656 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.595657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.596299 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.596301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.596733 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.596734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.596788 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.597185 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.597189 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.597544 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.599806 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.599807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.599807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.600026 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.600257 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.600257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.600258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.600323 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.600573 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.600573 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.600794 (+ 221us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.601379 (+ 585us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.601487 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":4,"thread_start_us":53,"threads_started":1}
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663349, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.190@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
I20260929 06:29:10.626583 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19531
I20260929 06:29:10.630060 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.603523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.0.0.1:52488)
0929 06:29:10.603575 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.603584 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.603617 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.603756 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.603757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.603906 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.603937 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.603938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.604486 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.604488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.604978 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.604980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.605297 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.605298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.605374 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.605778 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.605782 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.607272 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.628878 (+ 21606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.628879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.628880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.629152 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.629387 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.629388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.629388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.629448 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.629729 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.629730 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.629777 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.629841 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.629946 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 06:29:10.630434 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.603611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.13.15.188:42637)
0929 06:29:10.603665 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.603667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.603674 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:10.603699 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.603699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.603706 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.603745 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.604004 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.604412 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.605063 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.605233 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.605302 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.607351 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.607352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.607352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.607357 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.628782 (+ 21425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.629249 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.629250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.629251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.629313 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.629517 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.629518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.629518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.629642 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.629703 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.630278 (+ 575us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.630348 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:29:10.631285 19469 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.15.188:42637: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "70e84e810d524072820ef06b20d79427"
member_type: NON_VOTER
last_known_addr {
host: "127.13.15.188"
port: 42637
}
attrs {
promote: true
}
}
dest_uuid: "484ecb53ae9748e4b8848210efd2a507"
cas_config_opid_index: 5
I20260929 06:29:10.631508 19469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:10.632967 20028 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70e84e810d524072820ef06b20d79427 (127.13.15.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.634025 20028 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.15.188:42637
I20260929 06:29:10.634100 20028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.634197 20028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:10.634366 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:37749
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:38561
--webserver_port=41201
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:29:10.634651 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.13.15.189:37085)
0929 06:29:10.594351 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.594353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.598294 (+ 3941us) server_negotiation.cc:407] Connection header received
0929 06:29:10.598393 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.598394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.598404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.598434 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.627239 (+ 28805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.628036 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.630000 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.630332 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.630374 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.631459 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.631460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.631460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.631467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.633182 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.633747 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.633748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.633749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.633801 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.634004 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.634005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.634005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.634150 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.634197 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.634435 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.634515 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:29:10.634884 20163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:10.635044 20121 master.cc:537] Master@127.13.15.188:42637 shutting down...
I20260929 06:29:10.639360 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.632777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33754)
0929 06:29:10.635448 (+ 2671us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.635459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.635510 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.635832 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.635834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.635836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:10.635879 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.635881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.636758 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.636759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.638090 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.638094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.639078 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.639079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.639148 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.639169 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.639223 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.639267 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2654}
I20260929 06:29:10.639554 20163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 1 FOLLOWER]: Refusing update from remote peer 484ecb53ae9748e4b8848210efd2a507: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 06:29:10.639642 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.632860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.13.15.189:37085)
0929 06:29:10.634719 (+ 1859us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.634721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.635566 (+ 845us) server_negotiation.cc:407] Connection header received
0929 06:29:10.635589 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.635590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.635716 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.635775 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:10.635946 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.636670 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.638240 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.638896 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.638967 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.639042 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.639504 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.639544 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1844,"thread_start_us":53,"threads_started":1}
I20260929 06:29:10.639875 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.594476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.0.0.1:33738)
0929 06:29:10.598237 (+ 3761us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.598247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.598317 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.598499 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.598500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.626744 (+ 28244us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.627133 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.627137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.628884 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.628884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.629907 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.629909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.630463 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.630464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.630537 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.630962 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.630967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.631363 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.633435 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.633435 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.633436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.633662 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.633871 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.633872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.633872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.633941 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.634280 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.634281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.634320 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.634767 (+ 447us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.634875 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3729,"thread_start_us":64,"threads_started":1}
I20260929 06:29:10.640021 20077 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 LEARNER]: Refusing update from remote peer 484ecb53ae9748e4b8848210efd2a507: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 06:29:10.640189 20029 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:10.640479 20029 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 06:29:10.641144 20077 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70e84e810d524072820ef06b20d79427 (127.13.15.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
W20260929 06:29:10.641508 19453 consensus_peers.cc:487] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 -> Peer 70e84e810d524072820ef06b20d79427 (127.13.15.188:42637): Couldn't send request to peer 70e84e810d524072820ef06b20d79427. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 06:29:10.641724 20121 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:10.641803 20121 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:10.641848 20121 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: stopping tablet replica
I20260929 06:29:10.642485 20106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484ecb53ae9748e4b8848210efd2a507. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.642568 20106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: This master's current role is: LEARNER
I20260929 06:29:10.642616 20106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: NON_VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: true } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.642661 20106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: This master's current role is: LEARNER
I20260929 06:29:10.643049 20028 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: attempting to promote NON_VOTER 23abb0979d814a20bfbe50e0ebd31cce to VOTER
I20260929 06:29:10.643253 20028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:10.644016 20077 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 LEARNER]: Refusing update from remote peer 484ecb53ae9748e4b8848210efd2a507: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260929 06:29:10.644040 19453 consensus_peers.cc:487] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 -> Peer 70e84e810d524072820ef06b20d79427 (127.13.15.188:42637): Couldn't send request to peer 70e84e810d524072820ef06b20d79427. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 06:29:10.644172 20029 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 06:29:10.645427 20029 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 23abb0979d814a20bfbe50e0ebd31cce (127.13.15.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.646503 20029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.646593 20029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:10.646552 20028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.646638 20028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:10.646729 20077 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 23abb0979d814a20bfbe50e0ebd31cce (127.13.15.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.647735 20217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "484ecb53ae9748e4b8848210efd2a507" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.647830 20217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:29:10.656014 20121 master.cc:552] Master@127.13.15.188:42637 shutdown complete.
I20260929 06:29:10.656064 20121 master_runner.cc:305] Clearing existing system tablet
I20260929 06:29:10.656195 20121 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.656252 20121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.656970 20121 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.657480 20221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.657596 20121 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.657644 20121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.657668 20121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "70e84e810d524072820ef06b20d79427"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "3e0f74cee3ec20bc7c70040333df30a4"
server_key_iv: "e72a3a0de77f24518cdc1cf6bf376565"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.658473 20121 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:10.659940 20121 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 06:29:10.660007 20121 log.cc:1196] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 06:29:10.660187 20121 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Deleting consensus metadata
I20260929 06:29:10.660300 20121 master_runner.cc:315] Copying system tablet from 127.13.15.190:38561
I20260929 06:29:10.660912 20121 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: tablet copy: Beginning tablet copy session from remote peer at address 127.13.15.190:38561
I20260929 06:29:10.668406 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.661259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.13.15.190:38561)
0929 06:29:10.661304 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.661307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.661375 (+ 68us) server_negotiation.cc:407] Connection header received
0929 06:29:10.661439 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.661440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.661450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.661525 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.661860 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.662773 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.663762 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.664160 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.664244 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.665331 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.665333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.665333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.665341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.667105 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.667501 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.667503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.667503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.667568 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.667807 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.667808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.667808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.667950 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.668038 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.668223 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.668308 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:29:10.668439 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.661200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52140)
0929 06:29:10.661321 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.661332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.661388 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.661538 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.661539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.661738 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.661781 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.661783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.662869 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.662870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.663666 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.663669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.664260 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.664261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.664316 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.664849 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.664854 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.665224 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.667197 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.667198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.667199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.667417 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.667650 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.667651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.667651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.667727 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.668045 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.668046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.668098 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.668174 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.668346 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260929 06:29:10.669066 19489 tablet_copy_service.cc:140] P 484ecb53ae9748e4b8848210efd2a507: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 70e84e810d524072820ef06b20d79427 ({username='oryx', principal='oryx/127.13.15.188@KRBTEST.COM'} at 127.0.0.1:52140)
I20260929 06:29:10.669134 19489 tablet_copy_service.cc:161] P 484ecb53ae9748e4b8848210efd2a507: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 70e84e810d524072820ef06b20d79427 at {username='oryx', principal='oryx/127.13.15.188@KRBTEST.COM'} at 127.0.0.1:52140: session id = 70e84e810d524072820ef06b20d79427-00000000000000000000000000000000
I20260929 06:29:10.669734 19489 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 06:29:10.670231 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 06:29:10.671424 20121 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: tablet copy: Starting download of 0 data blocks...
I20260929 06:29:10.671561 20121 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: tablet copy: Starting download of 1 WAL segments...
I20260929 06:29:10.672400 20121 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 06:29:10.673218 19489 tablet_copy_service.cc:342] P 484ecb53ae9748e4b8848210efd2a507: Request end of tablet copy session 70e84e810d524072820ef06b20d79427-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.15.188@KRBTEST.COM'} at 127.0.0.1:52140
I20260929 06:29:10.673283 19489 tablet_copy_service.cc:433] P 484ecb53ae9748e4b8848210efd2a507: ending tablet copy session 70e84e810d524072820ef06b20d79427-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 70e84e810d524072820ef06b20d79427
I20260929 06:29:10.675658 20121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.676994 20228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.677048 20229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.677248 20231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.677374 20121 server_base.cc:1034] running on GCE node
I20260929 06:29:10.677634 20121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.677749 20121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.678859 20121 hybrid_clock.cc:648] HybridClock initialized: now 1790663350678850 us; error 25 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.680455 20121 init.cc:377] Logged in from keytab as oryx/127.13.15.188@KRBTEST.COM (short username oryx)
I20260929 06:29:10.680714 20121 webserver.cc:460] Webserver started at http://127.13.15.188:44169/ using document root <none> and password file <none>
I20260929 06:29:10.680858 20121 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.680888 20121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.681628 20121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:10.682237 20238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.682371 20121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.682415 20121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "70e84e810d524072820ef06b20d79427"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "3e0f74cee3ec20bc7c70040333df30a4"
server_key_iv: "e72a3a0de77f24518cdc1cf6bf376565"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.682485 20121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.700027 20121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.701964 20121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.704998 20121 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.188:42637
I20260929 06:29:10.705107 20290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.188:42637 every 8 connection(s)
I20260929 06:29:10.705403 20121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
I20260929 06:29:10.706346 20291 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:29:10.706867 20291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Bootstrap starting.
I20260929 06:29:10.709836 20291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:29:10.710074 20291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: Bootstrap complete.
I20260929 06:29:10.710290 20291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:10.710386 20291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70e84e810d524072820ef06b20d79427, State: Initialized, Role: LEARNER
I20260929 06:29:10.710426 20291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:10.710654 20291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:10.710649 20294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } } }
I20260929 06:29:10.710888 20294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427 [sys.catalog]: This master's current role is: LEARNER
I20260929 06:29:10.712184 20305 catalog_manager.cc:1259] Loaded cluster ID: 908d4c18b8ec42ecaa9a31f858c56dfc
I20260929 06:29:10.712417 20305 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: loading cluster ID for follower catalog manager: success
I20260929 06:29:10.714457 20305 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: acquiring CA information for follower catalog manager: success
I20260929 06:29:10.714917 20305 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70e84e810d524072820ef06b20d79427: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 06:29:10.734076 20215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:10.734377 20215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:10.734422 20215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:10.735719 20215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:10.735787 20215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:10.735838 20215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:10.737462 20215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:37749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=41201
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:10.737743 20215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:10.737987 20215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.743438 20310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.743428 20311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.743409 20313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.743718 20215 server_base.cc:1034] running on GCE node
I20260929 06:29:10.743928 20215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.744196 20215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.745373 20215 hybrid_clock.cc:648] HybridClock initialized: now 1790663350745351 us; error 37 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.748222 20215 init.cc:377] Logged in from keytab as oryx/127.13.15.129@KRBTEST.COM (short username oryx)
I20260929 06:29:10.748785 20215 webserver.cc:460] Webserver started at http://127.13.15.129:41201/ using document root <none> and password file <none>
I20260929 06:29:10.749032 20215 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.749119 20215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.750737 20215 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:29:10.751520 20320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.751765 20215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:10.751856 20215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "e715f40d0ed9491b963edc65cbc261af"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "c5b161d13424aa9025c774af0ec26ca0"
server_key_iv: "08376661a753c362706c310b3a42c4ad"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.752247 20215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.771764 20215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.773455 20215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:10.773581 20215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.773746 20215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:10.773965 20215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:10.773993 20215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.774013 20215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:10.774025 20215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.779428 20215 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:37749
I20260929 06:29:10.779485 20433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:37749 every 8 connection(s)
I20260929 06:29:10.779783 20215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
I20260929 06:29:10.789885 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20215
I20260929 06:29:10.790046 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19665
I20260929 06:29:10.790942 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:51811 (local address 127.13.15.189:37085)
0929 06:29:10.780591 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.780594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.781151 (+ 557us) server_negotiation.cc:407] Connection header received
0929 06:29:10.781908 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.781910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.781923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.781954 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.782775 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783491 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784446 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.784707 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784768 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.787599 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.787600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.787600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.787607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.789240 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789669 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.789670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.789670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.789727 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789943 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.789944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.789944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.790119 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.790198 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.790779 (+ 581us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.790852 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:29:10.791208 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.13.15.129:51811)
0929 06:29:10.781110 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.781116 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.781802 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.782098 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.782099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.782488 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.782674 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.782680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783577 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.783578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.784361 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.784363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.784781 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.784901 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.785353 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.785361 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.787518 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.789326 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.789328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.789600 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.789797 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.789798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.789876 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.790222 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.790227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.790634 (+ 407us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.790758 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.790910 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260929 06:29:10.791633 20436 heartbeater.cc:347] Connected to a master server at 127.13.15.189:37085
I20260929 06:29:10.791719 20436 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.791872 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:53941)
0929 06:29:10.781027 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.781036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.781793 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.782098 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.782100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.782488 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.782674 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.782677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783489 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.783490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.784523 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.784525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.784950 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.785013 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.785515 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.785520 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.787047 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.789283 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.789290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.789545 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.789746 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.789747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.789886 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.791361 (+ 1475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.791362 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.791404 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.791584 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.791696 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":39,"thread_start_us":75,"threads_started":1}
I20260929 06:29:10.791887 20436 heartbeater.cc:511] Master 127.13.15.189:37085 requested a full tablet report, sending...
I20260929 06:29:10.792136 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53941 (local address 127.13.15.190:38561)
0929 06:29:10.780873 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.780874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.781096 (+ 222us) server_negotiation.cc:407] Connection header received
0929 06:29:10.781905 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.781906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.781920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.781994 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.782768 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783392 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784604 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.784875 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784951 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.787141 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.787143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.787143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.787149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.789016 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789624 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.789625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.789626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.789687 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.789979 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.789981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.789981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.791260 (+ 1279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.791313 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.791957 (+ 644us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.792055 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 06:29:10.792392 20067 ts_manager.cc:194] Registered new tserver with Master: e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749)
I20260929 06:29:10.792888 20435 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:10.792946 20435 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.793005 20435 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:10.793326 19469 ts_manager.cc:194] Re-registered known tserver with Master: e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749)
I20260929 06:29:10.793848 19469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53941
I20260929 06:29:10.795900 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:43017
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:38561
--webserver_port=42741
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:29:10.795936 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:60851 (local address 127.13.15.188:42637)
0929 06:29:10.780783 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.780785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.780800 (+ 15us) server_negotiation.cc:407] Connection header received
0929 06:29:10.781943 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.781944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.781957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.782020 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.782775 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783763 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.784653 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.784930 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.785008 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.788152 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.788154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.788154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.788162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.794859 (+ 6697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.795174 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.795175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.795175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.795235 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.795423 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.795424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.795424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.795541 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.795615 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.795758 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.795839 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260929 06:29:10.796268 20438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.780457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.13.15.129:60851)
0929 06:29:10.780715 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.780751 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.781790 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.782094 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.782099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.782492 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.782674 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.782680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.783840 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.783841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.784593 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.784595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.785062 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.785063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.785128 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.785712 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.785731 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.788045 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.794942 (+ 6897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.794943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.794944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.795106 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.795284 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.795285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.795286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.795336 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.795608 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.795609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.795647 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.795733 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.795877 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":5,"thread_start_us":85,"threads_started":1}
I20260929 06:29:10.796648 20434 heartbeater.cc:347] Connected to a master server at 127.13.15.188:42637
I20260929 06:29:10.796705 20434 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.796774 20434 heartbeater.cc:511] Master 127.13.15.188:42637 requested a full tablet report, sending...
I20260929 06:29:10.797140 20255 ts_manager.cc:194] Registered new tserver with Master: e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749)
W20260929 06:29:10.875126 20442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:10.875301 20442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:10.875326 20442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:10.876389 20442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:10.876421 20442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:10.876447 20442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:10.877753 20442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:43017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=42741
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:10.877903 20442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:10.878079 20442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:10.883147 20448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.883136 20450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:10.883139 20447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:10.883322 20442 server_base.cc:1034] running on GCE node
I20260929 06:29:10.883517 20442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:10.883759 20442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:10.884915 20442 hybrid_clock.cc:648] HybridClock initialized: now 1790663350884955 us; error 89 us; skew 500 ppm
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:10.887301 20442 init.cc:377] Logged in from keytab as oryx/127.13.15.130@KRBTEST.COM (short username oryx)
I20260929 06:29:10.887729 20442 webserver.cc:460] Webserver started at http://127.13.15.130:42741/ using document root <none> and password file <none>
I20260929 06:29:10.887948 20442 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:10.888025 20442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:10.889243 20442 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:10.889792 20457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.889959 20442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:10.890049 20442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "290a56d518e84239aa611bfda9b693ee"
format_stamp: "Formatted at 2026-09-29 06:29:09 on dist-test-slave-sm0p"
server_key: "df1be8a6c8e67d9909e0e1cb6d1d684e"
server_key_iv: "58cb4fcdefdd5ee92caa4c984c5e2a94"
server_key_version: "encryptionkey@0"
I20260929 06:29:10.890414 20442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:10.899575 20442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:10.901225 20442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:10.901368 20442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:10.901559 20442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:10.901831 20442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:10.901882 20442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.901919 20442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:10.901945 20442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:10.907527 20442 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43017
I20260929 06:29:10.907569 20570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43017 every 8 connection(s)
I20260929 06:29:10.907903 20442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:10.910805 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20442
I20260929 06:29:10.910996 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19799
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
Sep 29 06:29:10 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663350, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
I20260929 06:29:10.920125 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:44441
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:38561
--webserver_port=34605
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:29:10.921729 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:41863 (local address 127.13.15.188:42637)
0929 06:29:10.908615 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.908628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.909144 (+ 516us) server_negotiation.cc:407] Connection header received
0929 06:29:10.909993 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.909994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.910009 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.910039 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.911003 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.911847 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913425 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.913724 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913764 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.918058 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.918059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.918060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.918067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.920003 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920724 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.920725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.920725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.920781 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920989 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.920990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.920990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.921133 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.921175 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.921548 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.921632 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:29:10.921730 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:58673 (local address 127.13.15.190:38561)
0929 06:29:10.908869 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.908871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.908881 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:29:10.910118 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.910119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.910150 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.910207 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.911005 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.911851 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913426 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.913723 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913764 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.918059 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.918060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.918061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.918068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.920302 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920832 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.920833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.920833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.920887 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.921078 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.921079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.921079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.921201 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.921241 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.921549 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.921639 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:29:10.921779 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:34571 (local address 127.13.15.189:37085)
0929 06:29:10.908850 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:10.908852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:10.908861 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:10.909973 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:10.909974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:10.909986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:10.910043 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:10.911827 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.912632 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913803 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.914155 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.914206 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.918193 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:10.918194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:10.918194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:10.918202 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:10.920176 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920644 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.920645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.920645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.920712 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920975 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:10.920976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:10.920977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:10.921116 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:10.921157 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:10.921553 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 06:29:10.921647 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:29:10.922415 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:58673)
0929 06:29:10.908708 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.908746 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.909855 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.910223 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.910223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.910576 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.910867 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.910884 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.911981 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.911981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.913324 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.913327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.913852 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.913997 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.914612 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.914641 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.917946 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.920510 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.920750 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.920958 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.921009 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.921282 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.921283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.921374 (+ 91us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.922051 (+ 677us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.922199 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":101,"thread_start_us":88,"threads_started":1}
I20260929 06:29:10.922421 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.13.15.130:34571)
0929 06:29:10.908717 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.908746 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.909842 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.910106 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.910110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.910556 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.910947 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.910950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.912754 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.912755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.913705 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.913708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.914286 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.914287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.914363 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.914819 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.914824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.918064 (+ 3240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.920268 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.920560 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.920780 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.920908 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.921204 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.921209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.921374 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.921903 (+ 529us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.922090 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":8,"spinlock_wait_cycles":7552,"thread_start_us":114,"threads_started":1}
I20260929 06:29:10.922418 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:10.908929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.13.15.130:41863)
0929 06:29:10.909065 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:10.909074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:10.909855 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:10.910169 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:10.910170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:10.910600 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:10.910867 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.910884 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.911982 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.911982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.913324 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:10.913327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:10.913851 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:10.913852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:10.913997 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:10.914682 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:10.914687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:10.917945 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:10.920097 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.920634 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.920856 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:10.920857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:10.920857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:10.920914 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:10.921223 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:10.921224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:10.921374 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 06:29:10.922069 (+ 695us) client_negotiation.cc:240] Negotiation successful
0929 06:29:10.922284 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":190,"thread_start_us":99,"threads_started":1}
I20260929 06:29:10.923012 20573 heartbeater.cc:347] Connected to a master server at 127.13.15.189:37085
I20260929 06:29:10.923100 20573 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.923224 20571 heartbeater.cc:347] Connected to a master server at 127.13.15.188:42637
I20260929 06:29:10.923277 20571 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.923282 20573 heartbeater.cc:511] Master 127.13.15.189:37085 requested a full tablet report, sending...
I20260929 06:29:10.923362 20571 heartbeater.cc:511] Master 127.13.15.188:42637 requested a full tablet report, sending...
I20260929 06:29:10.923530 20572 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:10.923578 20572 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:10.923723 20255 ts_manager.cc:194] Registered new tserver with Master: 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:10.923797 20067 ts_manager.cc:194] Registered new tserver with Master: 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:10.924060 20572 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:10.924319 19469 ts_manager.cc:194] Re-registered known tserver with Master: 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:10.924696 19469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:58673
W20260929 06:29:11.002511 20578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:11.002723 20578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:11.002760 20578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:11.003883 20578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:11.003947 20578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:11.003988 20578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:11.005502 20578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:44441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=34605
--tserver_master_addrs=127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:11.005784 20578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:11.006011 20578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:11.011224 20584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:11.011315 20578 server_base.cc:1034] running on GCE node
W20260929 06:29:11.011210 20583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:11.011413 20586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:11.011615 20578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:11.011801 20578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:11.012943 20578 hybrid_clock.cc:648] HybridClock initialized: now 1790663351012929 us; error 30 us; skew 500 ppm
Sep 29 06:29:11 dist-test-slave-sm0p krb5kdc[19424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663351, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:11.015239 20578 init.cc:377] Logged in from keytab as oryx/127.13.15.131@KRBTEST.COM (short username oryx)
I20260929 06:29:11.015645 20578 webserver.cc:460] Webserver started at http://127.13.15.131:34605/ using document root <none> and password file <none>
I20260929 06:29:11.015856 20578 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:11.015935 20578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:11.017192 20578 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:11.017807 20593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:11.018021 20578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:11.018160 20578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "b2b8f98c7cc041769103428c09d47a43"
format_stamp: "Formatted at 2026-09-29 06:29:10 on dist-test-slave-sm0p"
server_key: "1a6624ce610579a1bb6df5506b0759f5"
server_key_iv: "650828fd4ca2676fd788d622a10469c4"
server_key_version: "encryptionkey@0"
I20260929 06:29:11.018510 20578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:11.044561 20578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:11.046945 20578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:11.047118 20578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:11.047353 20578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:11.047644 20578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:11.047694 20578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:11.047729 20578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:11.047753 20578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:11.053274 20578 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:44441
I20260929 06:29:11.053371 20706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:44441 every 8 connection(s)
I20260929 06:29:11.053694 20578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:11.055740 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20578
Sep 29 06:29:11 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663351, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.189@KRBTEST.COM
Sep 29 06:29:11 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663351, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
Sep 29 06:29:11 dist-test-slave-sm0p krb5kdc[19424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663351, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.188@KRBTEST.COM
I20260929 06:29:11.066910 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.057712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.13.15.188:42637)
0929 06:29:11.058913 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.058914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.058920 (+ 6us) server_negotiation.cc:407] Connection header received
0929 06:29:11.058938 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.058939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.058948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.058996 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.059788 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.060590 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.061619 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.061919 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.061976 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.063429 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.063430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.063431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.063437 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.065448 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.065826 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.065827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.065828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.065881 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.066065 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.066066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.066066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.066204 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.066291 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.066700 (+ 409us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.066803 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1185,"thread_start_us":49,"threads_started":1}
I20260929 06:29:11.068142 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:60863)
0929 06:29:11.054671 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.054715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.055258 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.055524 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.055544 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.056204 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:11.056446 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.056454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057365 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.057366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.058764 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.058767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059250 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.059368 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.060031 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:11.060040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:11.062639 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:11.066354 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.066360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.066363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.066672 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.066879 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.066880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.066880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.067012 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.067472 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:11.067479 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:11.067650 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.067772 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.067933 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":157,"thread_start_us":85,"threads_started":1}
I20260929 06:29:11.068490 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:60863 (local address 127.13.15.190:38561)
0929 06:29:11.054701 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.054704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.054790 (+ 86us) server_negotiation.cc:407] Connection header received
0929 06:29:11.055378 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.055379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.055391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.055442 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.056759 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057253 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.058859 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059139 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059197 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.062738 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.062740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.062740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.062747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.065963 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.066751 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.066752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.066752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.066811 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.067238 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.067239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.067240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.067376 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.067442 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.068294 (+ 852us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.068376 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260929 06:29:11.068953 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.057921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.13.15.189:37085)
0929 06:29:11.057968 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.057970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.059878 (+ 1908us) server_negotiation.cc:407] Connection header received
0929 06:29:11.059980 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.059981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.059992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.060029 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.060729 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.061480 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.062567 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.062894 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.062941 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.064773 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.064774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.064774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.064781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.066470 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.067528 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.067529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.067529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.067594 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.067802 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.067803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.067803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.068197 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.068276 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.068753 (+ 477us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.068846 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:29:11.070070 20708 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38561
I20260929 06:29:11.070112 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:41089 (local address 127.13.15.189:37085)
0929 06:29:11.054592 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.054595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.054864 (+ 269us) server_negotiation.cc:407] Connection header received
0929 06:29:11.055378 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.055379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.055392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.055425 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.056606 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057372 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.058816 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059101 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059147 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.062923 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.062925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.062925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.062932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.067139 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069280 (+ 2141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.069281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.069281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.069339 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069553 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.069554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.069554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.069678 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.069720 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.069935 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.070007 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 06:29:11.070204 20708 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:11.070397 20708 heartbeater.cc:511] Master 127.13.15.190:38561 requested a full tablet report, sending...
I20260929 06:29:11.070474 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37085 (local address 127.13.15.131:41089)
0929 06:29:11.054808 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.054819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.055280 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.055964 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.055965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.056207 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:11.056446 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.056452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057462 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.057463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.058736 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.058737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059225 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.059365 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.060351 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:11.060355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:11.062834 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:11.068940 (+ 6106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.068941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.068941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.069187 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.069418 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.069419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.069480 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.069768 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:11.069769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:11.069816 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.070214 (+ 398us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.070322 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":140,"thread_start_us":86,"threads_started":1}
I20260929 06:29:11.070583 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:38459 (local address 127.13.15.188:42637)
0929 06:29:11.054738 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.054740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.054869 (+ 129us) server_negotiation.cc:407] Connection header received
0929 06:29:11.055382 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.055383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.055395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.055425 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.056607 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057364 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.058816 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059101 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059141 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.067727 (+ 8586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.067729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.067729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.067736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.069353 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069756 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.069758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.069758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.069814 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.070026 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.070027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.070027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.070184 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.070230 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.070431 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.070505 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 06:29:11.070808 19469 ts_manager.cc:194] Re-registered known tserver with Master: b2b8f98c7cc041769103428c09d47a43 (127.13.15.131:44441)
I20260929 06:29:11.070873 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.054398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:42637 (local address 127.13.15.131:38459)
0929 06:29:11.054786 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.054798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.055280 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.055541 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.055542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.056137 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:11.056446 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.056452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.057462 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.057463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.058719 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.058721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.059227 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.059227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.059365 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.060236 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:11.060241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:11.067636 (+ 7395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:11.069452 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.069454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.069662 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.069886 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:11.069887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:11.069887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:11.069952 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:11.070280 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:11.070280 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:11.070318 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.070664 (+ 346us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.070774 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":339,"thread_start_us":78,"threads_started":1}
I20260929 06:29:11.071223 20707 heartbeater.cc:347] Connected to a master server at 127.13.15.188:42637
I20260929 06:29:11.071244 20709 heartbeater.cc:347] Connected to a master server at 127.13.15.189:37085
I20260929 06:29:11.071270 20707 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:11.071276 19469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:60863
I20260929 06:29:11.071272 20709 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:11.071359 20707 heartbeater.cc:511] Master 127.13.15.188:42637 requested a full tablet report, sending...
I20260929 06:29:11.071403 20709 heartbeater.cc:511] Master 127.13.15.189:37085 requested a full tablet report, sending...
I20260929 06:29:11.071610 20255 ts_manager.cc:194] Registered new tserver with Master: b2b8f98c7cc041769103428c09d47a43 (127.13.15.131:44441)
I20260929 06:29:11.071647 20067 ts_manager.cc:194] Registered new tserver with Master: b2b8f98c7cc041769103428c09d47a43 (127.13.15.131:44441)
I20260929 06:29:11.072865 20721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.058255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52142 (local address 127.13.15.190:38561)
0929 06:29:11.062513 (+ 4258us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.062515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.063323 (+ 808us) server_negotiation.cc:407] Connection header received
0929 06:29:11.063419 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.063419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.063432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.063472 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.064158 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.064960 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.066002 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.066342 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.066382 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.067998 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.067999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.067999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.068005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.072006 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.072305 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.072306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.072306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.072353 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.072475 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.072476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.072476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.072565 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.072643 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.072752 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.072807 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4232,"thread_start_us":76,"threads_started":1}
I20260929 06:29:11.074193 19469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:29:11.075721 19469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:11.083168 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:44441 (local address 127.0.0.1:36910)
0929 06:29:11.079350 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.079360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.079398 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.079686 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.079687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.079689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:11.079729 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.079730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.081033 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.081034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.082129 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.082226 (+ 97us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.082953 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.082953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.083008 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.083012 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.083059 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.083099 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 06:29:11.083210 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.13.15.131:44441)
0929 06:29:11.079522 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.079526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.079538 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:29:11.079575 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.079576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.079595 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.079688 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:11.079808 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.080895 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.082302 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.082868 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.082949 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.083007 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.083088 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.083132 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260929 06:29:11.084091 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43017 (local address 127.0.0.1:54808)
0929 06:29:11.080065 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.080074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.080118 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.080537 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.080538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.080540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:11.080571 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.080572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.081448 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.081449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.082681 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.082684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.083860 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.083861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.083923 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.083927 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.083968 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.084011 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":68,"threads_started":1}
I20260929 06:29:11.084101 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:37749 (local address 127.0.0.1:39212)
0929 06:29:11.079524 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.079533 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.079574 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.080242 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.080243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.080245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:11.080276 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.080277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.081603 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.081604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.082727 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.082729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.083906 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.083907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.083959 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.083962 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.083990 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.084029 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260929 06:29:11.084095 20641 tablet_service.cc:1467] Processing CreateTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae (DEFAULT_TABLE table=test-table [id=f75ac86dcc4b40bd9e83f7b94c6a4daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:11.084424 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.13.15.129:37749)
0929 06:29:11.079976 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.079980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.079991 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:29:11.080073 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.080076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.080092 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.080173 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:11.080349 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.081400 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.082858 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.083798 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.084258 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.084322 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.084342 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.084369 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260929 06:29:11.084533 20641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d4657c5e0d48368565b64dbcc3c0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:11.084803 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.079997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.13.15.130:43017)
0929 06:29:11.080387 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.080390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.080403 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:29:11.080431 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.080433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.080444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.080542 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:11.080636 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.081367 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.082787 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.083773 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.083852 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.084643 (+ 791us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.084678 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.084724 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":76,"threads_started":1}
I20260929 06:29:11.085103 20368 tablet_service.cc:1467] Processing CreateTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae (DEFAULT_TABLE table=test-table [id=f75ac86dcc4b40bd9e83f7b94c6a4daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:11.085417 20368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d4657c5e0d48368565b64dbcc3c0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:11.087846 20732 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Bootstrap starting.
I20260929 06:29:11.088076 20733 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Bootstrap starting.
I20260929 06:29:11.088569 20732 tablet_bootstrap.cc:654] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:11.088716 20733 tablet_bootstrap.cc:654] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:11.088776 20505 tablet_service.cc:1467] Processing CreateTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae (DEFAULT_TABLE table=test-table [id=f75ac86dcc4b40bd9e83f7b94c6a4daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:11.088948 20732 log.cc:824] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:11.089041 20733 log.cc:824] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:11.089102 20505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d4657c5e0d48368565b64dbcc3c0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:11.089700 20732 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: No bootstrap required, opened a new log
I20260929 06:29:11.089744 20732 ts_tablet_manager.cc:1397] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:11.090545 20733 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: No bootstrap required, opened a new log
I20260929 06:29:11.090617 20733 ts_tablet_manager.cc:1397] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 06:29:11.091948 20736 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Bootstrap starting.
I20260929 06:29:11.092190 20732 raft_consensus.cc:348] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.092389 20732 raft_consensus.cc:374] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:11.092434 20732 raft_consensus.cc:729] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b8f98c7cc041769103428c09d47a43, State: Initialized, Role: FOLLOWER
I20260929 06:29:11.092553 20732 consensus_queue.cc:260] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.092701 20736 tablet_bootstrap.cc:654] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:11.092813 20732 ts_tablet_manager.cc:1428] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:29:11.092957 20708 heartbeater.cc:503] Master 127.13.15.190:38561 was elected leader, sending a full tablet report...
I20260929 06:29:11.092885 20733 raft_consensus.cc:348] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.093029 20733 raft_consensus.cc:374] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:11.093065 20733 raft_consensus.cc:729] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e715f40d0ed9491b963edc65cbc261af, State: Initialized, Role: FOLLOWER
I20260929 06:29:11.093053 20736 log.cc:824] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:11.093187 20733 consensus_queue.cc:260] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.094044 20733 ts_tablet_manager.cc:1428] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:29:11.094413 20435 heartbeater.cc:503] Master 127.13.15.190:38561 was elected leader, sending a full tablet report...
I20260929 06:29:11.095155 20736 tablet_bootstrap.cc:492] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: No bootstrap required, opened a new log
I20260929 06:29:11.095252 20736 ts_tablet_manager.cc:1397] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 06:29:11.096710 20736 raft_consensus.cc:348] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.096853 20736 raft_consensus.cc:374] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:11.096899 20736 raft_consensus.cc:729] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290a56d518e84239aa611bfda9b693ee, State: Initialized, Role: FOLLOWER
I20260929 06:29:11.097015 20736 consensus_queue.cc:260] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.097282 20736 ts_tablet_manager.cc:1428] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:29:11.097410 20572 heartbeater.cc:503] Master 127.13.15.190:38561 was elected leader, sending a full tablet report...
I20260929 06:29:11.158757 20737 raft_consensus.cc:482] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260929 06:29:11.158816 20574 tablet.cc:2369] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:11.158923 20737 raft_consensus.cc:504] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.159304 20737 leader_election.cc:288] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749), 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:11.163094 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.159579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:58351 (local address 127.13.15.130:43017)
0929 06:29:11.159618 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.159621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.159628 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:11.159649 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.159650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.159658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.159718 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:11.159851 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.160641 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162169 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.162737 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162805 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.162813 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.162992 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.163029 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:29:11.163094 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.159479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:57325 (local address 127.13.15.129:37749)
0929 06:29:11.159520 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.159522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.159531 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:11.159607 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.159608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.159619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.159680 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:11.159854 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.160647 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162165 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.162730 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162805 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.162813 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.162992 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.163029 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:29:11.163148 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.159487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43017 (local address 127.13.15.131:58351)
0929 06:29:11.159532 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.159553 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.159580 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.159728 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.159728 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.159729 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:11.159777 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.159778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.160726 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.160727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.162060 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.162064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.162814 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.162912 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.162915 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.162972 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.163024 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":70}
I20260929 06:29:11.163130 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.159430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:37749 (local address 127.13.15.131:57325)
0929 06:29:11.159478 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:11.159490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:11.159532 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:11.159688 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:11.159689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:11.159690 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:11.159777 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.159778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.160726 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.160726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.162060 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:11.162064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.162814 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.162815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:11.162912 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.162915 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:11.162972 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:29:11.163024 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":99}
I20260929 06:29:11.163537 20525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d4657c5e0d48368565b64dbcc3c0ae" candidate_uuid: "b2b8f98c7cc041769103428c09d47a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "290a56d518e84239aa611bfda9b693ee" is_pre_election: true
I20260929 06:29:11.163694 20525 raft_consensus.cc:2454] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b8f98c7cc041769103428c09d47a43 in term 0.
I20260929 06:29:11.163853 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d4657c5e0d48368565b64dbcc3c0ae" candidate_uuid: "b2b8f98c7cc041769103428c09d47a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715f40d0ed9491b963edc65cbc261af" is_pre_election: true
I20260929 06:29:11.163924 20594 leader_election.cc:302] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 290a56d518e84239aa611bfda9b693ee, b2b8f98c7cc041769103428c09d47a43; no voters:
I20260929 06:29:11.163997 20388 raft_consensus.cc:2454] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b8f98c7cc041769103428c09d47a43 in term 0.
I20260929 06:29:11.164044 20737 raft_consensus.cc:2781] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:11.164094 20737 raft_consensus.cc:482] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:11.164117 20737 raft_consensus.cc:3037] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:11.164783 20737 raft_consensus.cc:504] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.164909 20737 leader_election.cc:288] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [CANDIDATE]: Term 1 election: Requested vote from peers e715f40d0ed9491b963edc65cbc261af (127.13.15.129:37749), 290a56d518e84239aa611bfda9b693ee (127.13.15.130:43017)
I20260929 06:29:11.165040 20388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d4657c5e0d48368565b64dbcc3c0ae" candidate_uuid: "b2b8f98c7cc041769103428c09d47a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e715f40d0ed9491b963edc65cbc261af"
I20260929 06:29:11.165050 20525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d4657c5e0d48368565b64dbcc3c0ae" candidate_uuid: "b2b8f98c7cc041769103428c09d47a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "290a56d518e84239aa611bfda9b693ee"
I20260929 06:29:11.165117 20388 raft_consensus.cc:3037] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:11.165117 20525 raft_consensus.cc:3037] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:11.166004 20525 raft_consensus.cc:2454] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b8f98c7cc041769103428c09d47a43 in term 1.
I20260929 06:29:11.166004 20388 raft_consensus.cc:2454] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b8f98c7cc041769103428c09d47a43 in term 1.
I20260929 06:29:11.166190 20595 leader_election.cc:302] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b8f98c7cc041769103428c09d47a43, e715f40d0ed9491b963edc65cbc261af; no voters:
I20260929 06:29:11.166282 20737 raft_consensus.cc:2781] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:11.166415 20737 raft_consensus.cc:686] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 1 LEADER]: Becoming Leader. State: Replica: b2b8f98c7cc041769103428c09d47a43, State: Running, Role: LEADER
I20260929 06:29:11.166486 20737 consensus_queue.cc:237] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } }
I20260929 06:29:11.167245 19469 catalog_manager.cc:5495] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b8f98c7cc041769103428c09d47a43 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "b2b8f98c7cc041769103428c09d47a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b8f98c7cc041769103428c09d47a43" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 } health_report { overall_health: UNKNOWN } } }
I20260929 06:29:11.177601 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.175654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.13.15.131:44441)
0929 06:29:11.175692 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.175694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.175702 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:11.175751 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.175752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.175765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.175825 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:11.175924 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.176366 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.177031 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.177214 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.177260 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.177381 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:11.177463 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:11.177518 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.177545 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.177559 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:29:11.179100 20525 raft_consensus.cc:1261] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 1 FOLLOWER]: Refusing update from remote peer b2b8f98c7cc041769103428c09d47a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:11.179100 20388 raft_consensus.cc:1261] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 1 FOLLOWER]: Refusing update from remote peer b2b8f98c7cc041769103428c09d47a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:11.179380 20737 consensus_queue.cc:1035] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e715f40d0ed9491b963edc65cbc261af" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:11.179528 20741 consensus_queue.cc:1035] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "290a56d518e84239aa611bfda9b693ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:11.181525 20744 mvcc.cc:205] Tried to move back new op lower bound from 7334557086427451392 to 7334557086378565632. Current Snapshot: MvccSnapshot[applied={T|T < 7334557086427451392}]
I20260929 06:29:11.182104 20745 mvcc.cc:205] Tried to move back new op lower bound from 7334557086427451392 to 7334557086378565632. Current Snapshot: MvccSnapshot[applied={T|T < 7334557086427451392}]
I20260929 06:29:11.184262 19468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:11.184347 19468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:11.185578 19468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 484ecb53ae9748e4b8848210efd2a507: Sending DeleteTablet for 3 replicas of tablet 43d4657c5e0d48368565b64dbcc3c0ae
I20260929 06:29:11.185892 20368 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:11 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:39212
I20260929 06:29:11.185906 20505 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:11 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260929 06:29:11.186024 20641 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d4657c5e0d48368565b64dbcc3c0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:11 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:36910
I20260929 06:29:11.186225 20757 tablet_replica.cc:330] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: stopping tablet replica
I20260929 06:29:11.186232 20756 tablet_replica.cc:330] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: stopping tablet replica
I20260929 06:29:11.186309 20757 raft_consensus.cc:2229] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:11.186321 20756 raft_consensus.cc:2229] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:11.186408 20756 raft_consensus.cc:2258] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:11.186411 20757 raft_consensus.cc:2258] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:11.186540 20755 tablet_replica.cc:330] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: stopping tablet replica
I20260929 06:29:11.186625 20755 raft_consensus.cc:2229] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:11.186717 20755 raft_consensus.cc:2258] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:11.186987 20757 ts_tablet_manager.cc:1905] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:11.187088 20755 ts_tablet_manager.cc:1905] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:11.187463 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19437
I20260929 06:29:11.188933 20755 ts_tablet_manager.cc:1918] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:11.188943 20757 ts_tablet_manager.cc:1918] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:11.189006 20755 log.cc:1196] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/43d4657c5e0d48368565b64dbcc3c0ae
I20260929 06:29:11.189005 20757 log.cc:1196] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/43d4657c5e0d48368565b64dbcc3c0ae
I20260929 06:29:11.189261 20757 ts_tablet_manager.cc:1939] T 43d4657c5e0d48368565b64dbcc3c0ae P b2b8f98c7cc041769103428c09d47a43: Deleting consensus metadata
I20260929 06:29:11.189287 20755 ts_tablet_manager.cc:1939] T 43d4657c5e0d48368565b64dbcc3c0ae P 290a56d518e84239aa611bfda9b693ee: Deleting consensus metadata
I20260929 06:29:11.190457 20756 ts_tablet_manager.cc:1905] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:11.191709 20756 ts_tablet_manager.cc:1918] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:11.191846 20756 log.cc:1196] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/43d4657c5e0d48368565b64dbcc3c0ae
I20260929 06:29:11.192153 20756 ts_tablet_manager.cc:1939] T 43d4657c5e0d48368565b64dbcc3c0ae P e715f40d0ed9491b963edc65cbc261af: Deleting consensus metadata
W20260929 06:29:11.194753 20460 connection.cc:535] server connection from 127.0.0.1:54808 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 06:29:11.194855 20322 connection.cc:535] server connection from 127.0.0.1:39212 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 06:29:11.195057 20596 connection.cc:535] server connection from 127.0.0.1:36910 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 06:29:11.195135 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.187085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.13.15.188:42637)
0929 06:29:11.187138 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.187140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.187148 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:11.187168 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.187168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.187178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.187250 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.188067 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.188749 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.189926 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.190283 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.190350 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.192015 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.192016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.192016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.192023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.193862 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.194280 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.194281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.194282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.194335 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.194522 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.194523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.194524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.194664 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.194737 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.194949 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.195026 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:29:11.203449 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.196670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52528 (local address 127.13.15.188:42637)
0929 06:29:11.196708 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.196710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.196879 (+ 169us) server_negotiation.cc:407] Connection header received
0929 06:29:11.196975 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.196976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.196985 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.197057 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.197550 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.198284 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.199071 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.199312 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.199424 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.200747 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.200748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.200749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.200755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.202208 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.202644 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.202645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.202646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.202697 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.202918 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.202919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.202919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.203036 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.203103 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.203299 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.203360 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:29:11.204106 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:11.196812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.13.15.189:37085)
0929 06:29:11.196858 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:11.196860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:11.196973 (+ 113us) server_negotiation.cc:407] Connection header received
0929 06:29:11.197066 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:11.197068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:11.197077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:11.197105 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:11.197801 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.198542 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.199658 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:11.199967 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:11.200008 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:11.201380 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:11.201381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:11.201382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:11.201388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:11.203025 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.203391 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.203392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.203393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.203441 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:11.203622 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:11.203622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:11.203623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:11.203744 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:11.203806 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:11.203993 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 06:29:11.204050 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:29:12.182770 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:12.182476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:39293)
0929 06:29:12.182641 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:12.182718 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
W20260929 06:29:12.183127 20435 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38561 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:12.183627 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:12.183418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:39859)
0929 06:29:12.183511 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:12.183567 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":45,"threads_started":1}
W20260929 06:29:12.183838 20708 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38561 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:12.184336 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:12.184197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:58545)
0929 06:29:12.184267 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:12.184307 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60,"thread_start_us":34,"threads_started":1}
W20260929 06:29:12.184485 20572 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38561 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:12.685829 20768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:12.685961 20768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:12.686267 20768 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:12.686544 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:12.686372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52644)
0929 06:29:12.686462 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:12.686509 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":41,"threads_started":1}
W20260929 06:29:12.686677 20052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:12.687007 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:12.686922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:52650)
0929 06:29:12.686965 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:12.686989 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:29:12.687093 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:12.687141 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:12.687234 20768 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:13.183759 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:13.183520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:55777)
0929 06:29:13.183664 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:13.183718 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260929 06:29:13.184275 20772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:13.184101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:55109)
0929 06:29:13.184198 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:13.184233 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":65,"threads_started":1}
I20260929 06:29:13.184892 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:13.184751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:43421)
0929 06:29:13.184832 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:13.184861 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64,"thread_start_us":43,"threads_started":1}
I20260929 06:29:14.184508 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:14.184271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:35863)
0929 06:29:14.184418 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:14.184465 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260929 06:29:14.184906 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:14.184725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:49661)
0929 06:29:14.184826 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:14.184860 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
I20260929 06:29:14.185403 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:14.185252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:50087)
0929 06:29:14.185349 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:14.185372 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
I20260929 06:29:14.405869 20777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:14.405983 20777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:14.406199 20777 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:14.406509 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:14.406337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54318)
0929 06:29:14.406438 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:14.406474 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260929 06:29:14.406903 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:14.406810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54328)
0929 06:29:14.406855 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:14.406885 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:29:14.406971 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:14.407012 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:14.407083 20777 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:15.185180 20780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:15.184904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:57423)
0929 06:29:15.185014 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:15.185054 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260929 06:29:15.185446 20781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:15.185245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:49761)
0929 06:29:15.185358 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:15.185396 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260929 06:29:15.185869 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:15.185725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:52441)
0929 06:29:15.185816 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:15.185839 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":40,"threads_started":1}
I20260929 06:29:16.185950 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:16.185691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:40671)
0929 06:29:16.185848 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:16.185898 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260929 06:29:16.186120 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:16.185808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:36099)
0929 06:29:16.185930 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:16.185966 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260929 06:29:16.186393 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:16.186245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:47005)
0929 06:29:16.186332 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:16.186363 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":49,"threads_started":1}
I20260929 06:29:17.159192 20786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:17.159286 20786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:17.159420 20786 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:17.159699 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:17.159538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54604)
0929 06:29:17.159625 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:17.159664 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":42,"threads_started":1}
I20260929 06:29:17.160043 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:17.159967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54618)
0929 06:29:17.160005 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:17.160028 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 06:29:17.160120 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:17.160169 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:17.160246 20786 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:17.186501 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:17.186291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:43797)
0929 06:29:17.186410 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:17.186445 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260929 06:29:17.186685 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:17.186502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:33323)
0929 06:29:17.186596 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:17.186634 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":47,"threads_started":1}
I20260929 06:29:17.186879 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:17.186687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:40385)
0929 06:29:17.186812 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:17.186842 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260929 06:29:18.187141 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:18.186903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:39583)
0929 06:29:18.187023 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:18.187061 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260929 06:29:18.187197 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:18.187026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:48159)
0929 06:29:18.187125 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:18.187160 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260929 06:29:18.187374 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:18.187180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:33363)
0929 06:29:18.187309 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:18.187337 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260929 06:29:19.187815 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:19.187582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:55689)
0929 06:29:19.187742 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:19.187771 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":57,"threads_started":1}
I20260929 06:29:19.187816 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:19.187598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:55425)
0929 06:29:19.187717 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:19.187756 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":51,"threads_started":1}
I20260929 06:29:19.187990 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:19.187770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:48973)
0929 06:29:19.187856 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:19.187953 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":40,"threads_started":1}
I20260929 06:29:19.545220 20798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:19.545333 20798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:19.545533 20798 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:19.545866 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:19.545663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54802)
0929 06:29:19.545778 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:19.545829 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
W20260929 06:29:19.545971 20052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260929 06:29:19.546360 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:19.546263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:54804)
0929 06:29:19.546311 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:19.546336 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:29:19.546432 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:19.546474 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:19.546530 20798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:20.188452 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:20.188198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:54635)
0929 06:29:20.188352 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:20.188393 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":50,"threads_started":1}
I20260929 06:29:20.188473 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:20.188180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:35849)
0929 06:29:20.188389 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:20.188425 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":60,"threads_started":1}
I20260929 06:29:20.188637 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:20.188475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:39475)
0929 06:29:20.188568 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:20.188597 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260929 06:29:21.189105 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:21.188853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:40707)
0929 06:29:21.189004 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:21.189046 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260929 06:29:21.189196 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:21.189021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:41845)
0929 06:29:21.189117 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:21.189151 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
I20260929 06:29:21.189289 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:21.189133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:37363)
0929 06:29:21.189226 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:21.189256 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":45,"threads_started":1}
I20260929 06:29:22.189988 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:22.189733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:47951)
0929 06:29:22.189894 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:22.189933 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260929 06:29:22.189987 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:22.189757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:47703)
0929 06:29:22.189890 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:22.189932 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":52,"threads_started":1}
I20260929 06:29:22.190021 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:22.189813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:34571)
0929 06:29:22.189951 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:22.189979 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
I20260929 06:29:23.190764 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:23.190536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:56511)
0929 06:29:23.190667 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:23.190707 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260929 06:29:23.190851 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:23.190500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:45837)
0929 06:29:23.190649 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:23.190799 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260929 06:29:23.190986 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:23.190590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:55045)
0929 06:29:23.190906 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:23.190941 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":47,"threads_started":1}
I20260929 06:29:24.191421 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:24.191198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:58591)
0929 06:29:24.191330 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:24.191365 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260929 06:29:24.191557 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:24.191198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:51355)
0929 06:29:24.191473 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:24.191507 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
I20260929 06:29:24.191677 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:24.191500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:48885)
0929 06:29:24.191610 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:24.191638 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":48,"threads_started":1}
I20260929 06:29:24.511356 20816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:24.511464 20816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:24.511654 20816 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:24.511999 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:24.511792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:45116)
0929 06:29:24.511912 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:24.511962 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":44,"threads_started":1}
I20260929 06:29:24.512429 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:24.512326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:45124)
0929 06:29:24.512379 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:24.512405 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 06:29:24.512507 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:24.512547 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:24.512624 20816 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:25.192202 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:25.191962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:37699)
0929 06:29:25.192103 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:25.192146 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260929 06:29:25.192270 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:25.192109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:46419)
0929 06:29:25.192202 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:25.192229 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":41,"threads_started":1}
I20260929 06:29:25.192283 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:25.191962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:34489)
0929 06:29:25.192181 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:25.192218 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":59,"threads_started":1}
I20260929 06:29:26.192876 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:26.192605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:59399)
0929 06:29:26.192781 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:26.192822 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":50,"threads_started":1}
I20260929 06:29:26.192915 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:26.192704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:53589)
0929 06:29:26.192824 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:26.192861 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":44,"threads_started":1}
I20260929 06:29:26.193157 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:26.192577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:34433)
0929 06:29:26.193085 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:26.193127 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":58,"threads_started":1}
I20260929 06:29:26.721199 20825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:26.721289 20825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:26.721443 20825 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:26.721829 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:26.721623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:45256)
0929 06:29:26.721741 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:26.721790 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
W20260929 06:29:26.721917 20052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260929 06:29:26.722349 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:26.722221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:45264)
0929 06:29:26.722300 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:26.722330 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:29:26.722448 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:26.722499 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:26.722560 20825 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:27.193723 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:27.193502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:54845)
0929 06:29:27.193618 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:27.193656 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":47,"threads_started":1}
I20260929 06:29:27.193742 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:27.193353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:42699)
0929 06:29:27.193656 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:27.193694 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":59,"threads_started":1}
I20260929 06:29:27.193778 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:27.193624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:45831)
0929 06:29:27.193713 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:27.193737 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":43,"threads_started":1}
I20260929 06:29:28.194401 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:28.194111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:37169)
0929 06:29:28.194309 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:28.194346 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260929 06:29:28.194519 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:28.194111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:43251)
0929 06:29:28.194429 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:28.194481 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":75,"threads_started":1}
I20260929 06:29:28.194546 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:28.194167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:40193)
0929 06:29:28.194424 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:28.194477 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":56,"threads_started":1}
I20260929 06:29:29.195075 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:29.194848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:48225)
0929 06:29:29.194976 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:29.195008 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260929 06:29:29.195181 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:29.194942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:56943)
0929 06:29:29.195097 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:29.195130 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":54,"threads_started":1}
I20260929 06:29:29.195214 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:29.194921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:48243)
0929 06:29:29.195148 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:29.195180 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":39,"threads_started":1}
I20260929 06:29:30.195886 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:30.195560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:58337)
0929 06:29:30.195732 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:30.195783 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260929 06:29:30.195941 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:30.195679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:45843)
0929 06:29:30.195878 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:30.195904 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":40,"threads_started":1}
I20260929 06:29:30.195961 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:30.195617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:49005)
0929 06:29:30.195834 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:30.195884 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260929 06:29:31.196530 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:31.196291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:55531)
0929 06:29:31.196420 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:31.196471 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260929 06:29:31.196518 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:31.196320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:33375)
0929 06:29:31.196413 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:31.196469 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":45,"threads_started":1}
I20260929 06:29:31.196519 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:31.196291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:49743)
0929 06:29:31.196420 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:31.196470 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260929 06:29:32.197216 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:32.196990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:43361)
0929 06:29:32.197132 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:32.197167 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260929 06:29:32.197218 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:32.196996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:37335)
0929 06:29:32.197120 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:32.197164 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260929 06:29:32.197316 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:32.197034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:40369)
0929 06:29:32.197234 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:32.197272 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":56,"threads_started":1}
I20260929 06:29:33.197896 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:33.197687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:42005)
0929 06:29:33.197810 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:33.197845 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260929 06:29:33.197896 20848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:33.197684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:42493)
0929 06:29:33.197810 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:33.197845 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260929 06:29:33.197954 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:33.197687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:45277)
0929 06:29:33.197842 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:33.197882 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":59,"threads_started":1}
I20260929 06:29:34.198539 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:34.198322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:51673)
0929 06:29:34.198481 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:34.198506 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":64,"threads_started":1}
I20260929 06:29:34.198534 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:34.198322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:33749)
0929 06:29:34.198443 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:34.198481 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260929 06:29:34.198719 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:34.198471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:40177)
0929 06:29:34.198569 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:34.198622 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":52,"threads_started":1}
I20260929 06:29:35.199158 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:35.198905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:55889)
0929 06:29:35.199061 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:35.199103 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260929 06:29:35.199178 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:35.198893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:33499)
0929 06:29:35.199111 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:35.199142 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
I20260929 06:29:35.199278 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:35.199128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:54731)
0929 06:29:35.199212 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:35.199238 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":39,"threads_started":1}
I20260929 06:29:36.199855 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:36.199638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:33841)
0929 06:29:36.199756 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:36.199794 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":47,"threads_started":1}
I20260929 06:29:36.199936 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:36.199622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:60077)
0929 06:29:36.199868 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:36.199898 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":64,"threads_started":1}
I20260929 06:29:36.200063 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:36.199819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:50291)
0929 06:29:36.199957 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:36.199998 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
I20260929 06:29:37.191238 20858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 484ecb53ae9748e4b8848210efd2a507)
I20260929 06:29:37.191337 20858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "484ecb53ae9748e4b8848210efd2a507" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38561 } } peers { permanent_uuid: "23abb0979d814a20bfbe50e0ebd31cce" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37085 } attrs { promote: false } } peers { permanent_uuid: "70e84e810d524072820ef06b20d79427" member_type: NON_VOTER last_known_addr { host: "127.13.15.188" port: 42637 } attrs { promote: true } }
I20260929 06:29:37.191455 20858 leader_election.cc:288] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561)
I20260929 06:29:37.191847 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:37.191645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:39204)
0929 06:29:37.191760 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:37.191808 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
W20260929 06:29:37.191941 20052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:37.192318 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:37.192226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.0.0.1:39212)
0929 06:29:37.192272 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:37.192299 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:29:37.192471 20052 leader_election.cc:334] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 484ecb53ae9748e4b8848210efd2a507 (127.13.15.190:38561): Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
I20260929 06:29:37.192529 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 23abb0979d814a20bfbe50e0ebd31cce; no voters: 484ecb53ae9748e4b8848210efd2a507
I20260929 06:29:37.192628 20858 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 23abb0979d814a20bfbe50e0ebd31cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:29:37.200583 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:37.200383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:35015)
0929 06:29:37.200494 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:37.200529 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260929 06:29:37.200811 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:37.200590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:45591)
0929 06:29:37.200722 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:37.200762 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260929 06:29:37.200906 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:37.200581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:47371)
0929 06:29:37.200802 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:37.200838 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":52,"threads_started":1}
I20260929 06:29:38.201417 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:38.201195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:42903)
0929 06:29:38.201321 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:38.201361 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
I20260929 06:29:38.201413 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:38.201046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:46735)
0929 06:29:38.201331 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:38.201374 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":87,"threads_started":1}
I20260929 06:29:38.201646 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:38.201334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:41621)
0929 06:29:38.201566 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:38.201601 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":42,"threads_started":1}
I20260929 06:29:39.202253 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:39.201854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:48187)
0929 06:29:39.202119 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:39.202193 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":71,"threads_started":1}
I20260929 06:29:39.202314 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:39.202032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:55915)
0929 06:29:39.202231 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:39.202269 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":46,"threads_started":1}
I20260929 06:29:39.202364 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:39.202032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:50993)
0929 06:29:39.202267 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:39.202310 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":67,"threads_started":1}
I20260929 06:29:40.203052 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:40.202758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:59113)
0929 06:29:40.202955 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:40.202996 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":61,"threads_started":1}
I20260929 06:29:40.203074 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:40.202791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:53103)
0929 06:29:40.203005 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:40.203034 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":60,"threads_started":1}
I20260929 06:29:40.203083 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:40.202885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:36097)
0929 06:29:40.202979 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:40.203017 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":54,"threads_started":1}
I20260929 06:29:41.203871 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:41.203499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.130:33919)
0929 06:29:41.203754 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:41.203797 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":129,"threads_started":1}
I20260929 06:29:41.203903 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:41.203582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.129:33815)
0929 06:29:41.203822 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:41.203860 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":45,"threads_started":1}
I20260929 06:29:41.203951 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:41.203679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38561 (local address 127.13.15.131:53867)
0929 06:29:41.203833 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:41.203860 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.13.15.190:38561,127.13.15.189:37085,127.13.15.188:42637, num_attempts: 1)
I20260929 06:29:41.362614 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20215
I20260929 06:29:41.367699 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20442
I20260929 06:29:41.372602 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20578
I20260929 06:29:41.377918 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 19933
I20260929 06:29:41.383033 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20121
2026-09-29T06:29:41Z chronyd exiting
I20260929 06:29:41.399166 13374 test_util.cc:175] -----------------------------------------------
I20260929 06:29:41.399266 13374 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790663323716959-13374-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (32991 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:41 dist-test-slave-sm0p krb5kdc[20878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:41 dist-test-slave-sm0p krb5kdc[20878](info): set up 2 sockets
Sep 29 06:29:41 dist-test-slave-sm0p krb5kdc[20878](info): commencing operation
krb5kdc: starting...
W20260929 06:29:42.442865 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab.
Entry for principal oryx/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab.
I20260929 06:29:42.558723 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37973
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab
--principal=oryx/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:42.638705 20891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:42.638942 20891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:42.638978 20891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:42.640139 20891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:42.640219 20891 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 06:29:42.640241 20891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:42.640262 20891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:42.640297 20891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:42.640321 20891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:42.641829 20891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:37973
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.190.keytab
--principal=oryx/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:42.642154 20891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:42.642379 20891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:42.647053 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.647027 20899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.647066 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:42.647112 20891 server_base.cc:1034] running on GCE node
I20260929 06:29:42.647462 20891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:42.647759 20891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:42.648955 20891 hybrid_clock.cc:648] HybridClock initialized: now 1790663382648935 us; error 38 us; skew 500 ppm
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:42.651489 20891 init.cc:377] Logged in from keytab as oryx/127.13.15.190@KRBTEST.COM (short username oryx)
I20260929 06:29:42.651929 20891 webserver.cc:460] Webserver started at http://127.13.15.190:32905/ using document root <none> and password file <none>
I20260929 06:29:42.652169 20891 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:42.652235 20891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:42.652384 20891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:42.653738 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "7e0f265c1843490480a37c97b7ef30ae"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "a54f76aa73fc10b2e9e5eeef06a42e84"
server_key_iv: "d18ce1a6a86fa93d535cba3f21af40de"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.654208 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "7e0f265c1843490480a37c97b7ef30ae"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "a54f76aa73fc10b2e9e5eeef06a42e84"
server_key_iv: "d18ce1a6a86fa93d535cba3f21af40de"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.655948 20891 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:42.656744 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:42.656894 20891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:42.656999 20891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "7e0f265c1843490480a37c97b7ef30ae"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "a54f76aa73fc10b2e9e5eeef06a42e84"
server_key_iv: "d18ce1a6a86fa93d535cba3f21af40de"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.657051 20891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:42.665585 20891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:42.667471 20891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:42.667577 20891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:42.671921 20891 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:37973
I20260929 06:29:42.671995 20968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:37973 every 8 connection(s)
I20260929 06:29:42.672338 20891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:42.672806 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20891
I20260929 06:29:42.672933 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:42.673084 20969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:42.673146 13374 external_mini_cluster.cc:1428] Setting key 8f655c8059d63a98c3cfc4c52c8e04ae
I20260929 06:29:42.676440 20969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Bootstrap starting.
I20260929 06:29:42.677115 20969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:42.677400 20969 log.cc:824] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:42.678085 20969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: No bootstrap required, opened a new log
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:42.680624 20969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } }
I20260929 06:29:42.680780 20969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:42.680825 20969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0f265c1843490480a37c97b7ef30ae, State: Initialized, Role: FOLLOWER
I20260929 06:29:42.681002 20969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } }
I20260929 06:29:42.681102 20969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:42.681164 20969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:42.681257 20969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:42.682018 20969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } }
I20260929 06:29:42.682176 20969 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0f265c1843490480a37c97b7ef30ae; no voters:
I20260929 06:29:42.682344 20969 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:42.682397 20974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:42.682578 20969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:42.682582 20974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [term 1 LEADER]: Becoming Leader. State: Replica: 7e0f265c1843490480a37c97b7ef30ae, State: Running, Role: LEADER
I20260929 06:29:42.682775 20974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } }
I20260929 06:29:42.683095 20975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e0f265c1843490480a37c97b7ef30ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } } }
I20260929 06:29:42.683215 20976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0f265c1843490480a37c97b7ef30ae. Latest consensus state: current_term: 1 leader_uuid: "7e0f265c1843490480a37c97b7ef30ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0f265c1843490480a37c97b7ef30ae" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 37973 } } }
I20260929 06:29:42.683302 20975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:42.683373 20976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:42.683549 20983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:42.684098 20983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:42.684902 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:42.673665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.13.15.190:37973)
0929 06:29:42.673893 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:42.673901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:42.673932 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:29:42.674717 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:42.674734 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:42.674798 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:42.674948 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:42.675537 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.676915 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.677711 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.678235 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.678337 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:42.680923 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:42.680927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:42.680928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:42.680938 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:42.682974 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.683424 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.683428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.683429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.683533 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.683770 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.683771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.683772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.684147 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:42.684315 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:42.684544 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 06:29:42.684679 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":58,"threads_started":1}
I20260929 06:29:42.685743 20983 catalog_manager.cc:1347] Generated new cluster ID: 1a49cade20da4d5db3d718bda40e7998
I20260929 06:29:42.685793 20983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:42.699813 20983 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:42.700353 20983 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:42.708022 20983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Generated new TSK 0
I20260929 06:29:42.708237 20983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
I20260929 06:29:42.739339 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37973
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:42.822237 20995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:42.822459 20995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:42.822486 20995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:42.823730 20995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:42.823794 20995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:42.823874 20995 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:29:42.823922 20995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:42.825464 20995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:42.825742 20995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:42.825999 20995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:42.831053 21001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.831058 21000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.831095 21003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:42.831389 20995 server_base.cc:1034] running on GCE node
I20260929 06:29:42.831631 20995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:42.831874 20995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:42.833066 20995 hybrid_clock.cc:648] HybridClock initialized: now 1790663382833039 us; error 37 us; skew 500 ppm
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:42.839219 20995 init.cc:377] Logged in from keytab as oryx/127.13.15.129@KRBTEST.COM (short username oryx)
I20260929 06:29:42.839747 20995 webserver.cc:460] Webserver started at http://127.13.15.129:45673/ using document root <none> and password file <none>
I20260929 06:29:42.839984 20995 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:42.840075 20995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:42.840240 20995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:42.841302 20995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "2dc526a0efba4786a173766eaa1227d0"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "152cbc4ebab9a81bf122242a32af76b9"
server_key_iv: "165356971b4e152b71ba55376df53f0c"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.841692 20995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "2dc526a0efba4786a173766eaa1227d0"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "152cbc4ebab9a81bf122242a32af76b9"
server_key_iv: "165356971b4e152b71ba55376df53f0c"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.843180 20995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:42.843952 21010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:42.844144 20995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:42.844223 20995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "2dc526a0efba4786a173766eaa1227d0"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "152cbc4ebab9a81bf122242a32af76b9"
server_key_iv: "165356971b4e152b71ba55376df53f0c"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.844277 20995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:42.853201 20995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:42.855005 20995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:42.855118 20995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:42.855614 20995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:42.855677 20995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:42.855723 20995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:42.855769 20995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:42.863539 20995 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:42045
I20260929 06:29:42.863615 21125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:42045 every 8 connection(s)
I20260929 06:29:42.864034 20995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:42.864426 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 20995
I20260929 06:29:42.864527 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:42.864681 13374 external_mini_cluster.cc:1428] Setting key 3f06966490938231db080e0018855c93
I20260929 06:29:42.865923 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:42.855966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:57603 (local address 127.13.15.190:37973)
0929 06:29:42.856015 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:42.856018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:42.856260 (+ 242us) server_negotiation.cc:407] Connection header received
0929 06:29:42.856732 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:42.856733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:42.856747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:42.856844 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:42.857472 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.858272 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.859316 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.859697 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.859767 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:42.862597 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:42.862599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:42.862599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:42.862607 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:42.864396 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.864887 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.864888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.864889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.864980 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.865244 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.865245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.865245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.865372 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:42.865456 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:42.865739 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 06:29:42.865824 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 06:29:42.866096 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:42.855959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.129:57603)
0929 06:29:42.856113 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:42.856151 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:42.856640 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:42.856888 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:42.856891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:42.857163 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:42.857375 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:42.857380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.858364 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.858365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:42.859219 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:42.859221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.859804 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.859806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:42.859967 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:42.860652 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:42.860662 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:42.862489 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:42.864506 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:42.864511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:42.864513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:42.864759 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:42.865049 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:42.865050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:42.865050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:42.865166 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:42.865446 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:42.865452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:42.865593 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 06:29:42.865714 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 06:29:42.865869 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":92,"threads_started":1}
I20260929 06:29:42.866953 21126 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37973
I20260929 06:29:42.867029 21126 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:42.867249 21126 heartbeater.cc:511] Master 127.13.15.190:37973 requested a full tablet report, sending...
I20260929 06:29:42.867803 20922 ts_manager.cc:194] Registered new tserver with Master: 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045)
I20260929 06:29:42.868808 20922 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:57603
WARNING: no policy specified for oryx/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:29:42.874341 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:42.867940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:36197 (local address 127.13.15.190:37973)
0929 06:29:42.867976 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:42.867979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:42.868215 (+ 236us) server_negotiation.cc:407] Connection header received
0929 06:29:42.868277 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:42.868277 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:42.868287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:42.868338 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:42.868662 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.869354 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.870549 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.870728 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.870785 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:42.871824 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:42.871825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:42.871825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:42.871830 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:42.873141 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.873601 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.873601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.873602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.873650 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:42.873869 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:42.873870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:42.873870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:42.873959 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:42.874016 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:42.874233 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 06:29:42.874285 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:29:42.874372 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:42.867984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.129:36197)
0929 06:29:42.868121 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:42.868131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:42.868220 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:42.868344 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:42.868344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:42.868519 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:42.868571 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:42.868572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.869466 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.869467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:42.870459 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:42.870462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:42.870811 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:42.870812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:42.870890 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:42.871290 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:42.871296 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:42.871730 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:42.873227 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:42.873228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:42.873228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:42.873456 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:42.873727 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:42.873728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:42.873728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:42.873800 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:42.874029 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:42.874029 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:42.874064 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 06:29:42.874214 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 06:29:42.874305 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
I20260929 06:29:42.891041 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37973
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:42.971905 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:42.972093 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:42.972116 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:42.973278 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:42.973320 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:42.973341 21135 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:29:42.973407 21135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:42.974905 21135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:42.975136 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:42.975351 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:42.980062 21140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.980068 21143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:42.980068 21141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:42.980355 21135 server_base.cc:1034] running on GCE node
I20260929 06:29:42.980579 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:42.980846 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:42.981985 21135 hybrid_clock.cc:648] HybridClock initialized: now 1790663382981970 us; error 35 us; skew 500 ppm
Sep 29 06:29:42 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:42.984571 21135 init.cc:377] Logged in from keytab as oryx/127.13.15.130@KRBTEST.COM (short username oryx)
I20260929 06:29:42.985003 21135 webserver.cc:460] Webserver started at http://127.13.15.130:39849/ using document root <none> and password file <none>
I20260929 06:29:42.985239 21135 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:42.985283 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:42.985402 21135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:42.986519 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "cb76ad97af27428ba2cd775a48487b72"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "864ebe545a78b63280403970d5055d69"
server_key_iv: "09faf17abf3fb3e67b76003a94e12a4e"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.986866 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "cb76ad97af27428ba2cd775a48487b72"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "864ebe545a78b63280403970d5055d69"
server_key_iv: "09faf17abf3fb3e67b76003a94e12a4e"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.988134 21135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:42.988824 21150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:42.989006 21135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:42.989084 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "cb76ad97af27428ba2cd775a48487b72"
format_stamp: "Formatted at 2026-09-29 06:29:42 on dist-test-slave-sm0p"
server_key: "864ebe545a78b63280403970d5055d69"
server_key_iv: "09faf17abf3fb3e67b76003a94e12a4e"
server_key_version: "encryptionkey@0"
I20260929 06:29:42.989132 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:43.010885 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:43.012822 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:43.012933 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:43.013389 21135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:43.013432 21135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.013455 21135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:43.013469 21135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.020515 21135 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:32781
I20260929 06:29:43.020612 21265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:32781 every 8 connection(s)
I20260929 06:29:43.020946 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:43.024446 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.013901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:49711 (local address 127.13.15.190:37973)
0929 06:29:43.013978 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.013981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.014157 (+ 176us) server_negotiation.cc:407] Connection header received
0929 06:29:43.014721 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.014723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.014737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.014823 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.015699 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.016582 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.018218 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.018551 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.018640 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.022049 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.022051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.022051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.022057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.023266 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.023644 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.023645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.023646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.023695 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.023873 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.023874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.023874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.023973 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.024025 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.024324 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.024384 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:29:43.024674 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.013853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.130:49711)
0929 06:29:43.013999 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.014038 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.014610 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.014886 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.014889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.015269 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:43.015564 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.015571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.016723 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.016724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.018118 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.018121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.018640 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.018640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.018744 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.020198 (+ 1454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:43.020206 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:43.021964 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:43.023353 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.023358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.023360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.023572 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.023736 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.023737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.023737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.023824 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.024060 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:43.024066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:43.024197 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.024297 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.024415 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260929 06:29:43.025117 21266 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37973
I20260929 06:29:43.025204 21266 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:43.025408 21266 heartbeater.cc:511] Master 127.13.15.190:37973 requested a full tablet report, sending...
I20260929 06:29:43.025806 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21135
I20260929 06:29:43.025933 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:43.026124 13374 external_mini_cluster.cc:1428] Setting key ac64947e70529c18aa6a135aff2f7743
I20260929 06:29:43.026467 20923 ts_manager.cc:194] Registered new tserver with Master: cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:43.026973 20923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:49711
I20260929 06:29:43.032370 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.026207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:50453 (local address 127.13.15.190:37973)
0929 06:29:43.026245 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.026247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.026401 (+ 154us) server_negotiation.cc:407] Connection header received
0929 06:29:43.026451 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.026452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.026461 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.026516 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.026855 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.027529 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.028525 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.028718 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.028774 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.029808 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.029809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.029810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.029815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.031240 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.031637 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.031638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.031638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.031682 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.031906 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.031906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.031907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.032026 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.032074 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.032259 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.032314 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:29:43.032493 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.026137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.130:50453)
0929 06:29:43.026308 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.026319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.026401 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.026519 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.026520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.026691 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:43.026768 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.026770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.027638 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.027638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.028440 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.028443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.028794 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.028795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.028869 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.029292 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:43.029297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:43.029726 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:43.031324 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.031326 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.031326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.031562 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.031759 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.031760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.031761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.031840 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.032091 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:43.032092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:43.032154 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.032242 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.032357 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
I20260929 06:29:43.053422 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:37973
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:43.135128 21275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:43.135319 21275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:43.135341 21275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:43.136439 21275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:43.136477 21275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:43.136497 21275 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 06:29:43.136513 21275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:43.137955 21275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.15.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:43.138226 21275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:43.138463 21275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:43.143364 21281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:43.143342 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:43.143388 21275 server_base.cc:1034] running on GCE node
W20260929 06:29:43.143354 21283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:43.143739 21275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:43.143972 21275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:43.145130 21275 hybrid_clock.cc:648] HybridClock initialized: now 1790663383145116 us; error 45 us; skew 500 ppm
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:43.147763 21275 init.cc:377] Logged in from keytab as oryx/127.13.15.131@KRBTEST.COM (short username oryx)
I20260929 06:29:43.148171 21275 webserver.cc:460] Webserver started at http://127.13.15.131:46407/ using document root <none> and password file <none>
I20260929 06:29:43.148415 21275 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:43.148495 21275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:43.148653 21275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:43.149659 21275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "4dbf593401a64172976745e95e780011"
format_stamp: "Formatted at 2026-09-29 06:29:43 on dist-test-slave-sm0p"
server_key: "043f5ef3f1caa9ee096c51ee8f4cfb63"
server_key_iv: "88b0447dc5e0656d1069b41968224b89"
server_key_version: "encryptionkey@0"
I20260929 06:29:43.150014 21275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "4dbf593401a64172976745e95e780011"
format_stamp: "Formatted at 2026-09-29 06:29:43 on dist-test-slave-sm0p"
server_key: "043f5ef3f1caa9ee096c51ee8f4cfb63"
server_key_iv: "88b0447dc5e0656d1069b41968224b89"
server_key_version: "encryptionkey@0"
I20260929 06:29:43.151259 21275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:43.151924 21290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.152093 21275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.152189 21275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "4dbf593401a64172976745e95e780011"
format_stamp: "Formatted at 2026-09-29 06:29:43 on dist-test-slave-sm0p"
server_key: "043f5ef3f1caa9ee096c51ee8f4cfb63"
server_key_iv: "88b0447dc5e0656d1069b41968224b89"
server_key_version: "encryptionkey@0"
I20260929 06:29:43.152274 21275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:43.167026 21275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:43.169296 21275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:43.169458 21275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:43.169945 21275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:43.170008 21275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.170042 21275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:43.170070 21275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.176854 21275 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:44383
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:43.176955 21405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:44383 every 8 connection(s)
I20260929 06:29:43.177229 21275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:29:43.178081 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21275
I20260929 06:29:43.178241 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:43.178442 13374 external_mini_cluster.cc:1428] Setting key 2e1574d9dbe083c423467bc4a566d149
I20260929 06:29:43.180135 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.170405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:42501 (local address 127.13.15.190:37973)
0929 06:29:43.170458 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.170460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.170685 (+ 225us) server_negotiation.cc:407] Connection header received
0929 06:29:43.171161 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.171163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.171176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.171255 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.172051 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.172914 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.174454 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.174681 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.174762 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.177376 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.177377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.177378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.177385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.178903 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.179243 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.179244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.179245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.179312 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.179602 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.179603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.179603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.179749 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.179801 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.180019 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.180076 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:29:43.180276 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.170411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.131:42501)
0929 06:29:43.170551 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.170587 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.171071 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.171342 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.171347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.171671 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:43.171950 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.171957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.173008 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.173009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.174378 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.174382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.174781 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.174782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.174939 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.175470 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:43.175477 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:43.177279 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:43.178976 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.178981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.178982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.179180 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.179377 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.179378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.179378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.179515 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.179795 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:43.179800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:43.179912 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.179993 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.180100 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260929 06:29:43.180677 21406 heartbeater.cc:347] Connected to a master server at 127.13.15.190:37973
I20260929 06:29:43.180789 21406 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:43.180956 21406 heartbeater.cc:511] Master 127.13.15.190:37973 requested a full tablet report, sending...
I20260929 06:29:43.181298 20923 ts_manager.cc:194] Registered new tserver with Master: 4dbf593401a64172976745e95e780011 (127.13.15.131:44383)
I20260929 06:29:43.181797 20923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:42501
I20260929 06:29:43.187070 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.181687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:40333 (local address 127.13.15.190:37973)
0929 06:29:43.181723 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.181725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.181816 (+ 91us) server_negotiation.cc:407] Connection header received
0929 06:29:43.181868 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.181869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.181878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.181927 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.182243 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.182649 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.183323 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.183541 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.183613 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.184663 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.184664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.184664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.184670 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.185903 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.186310 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.186311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.186312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.186375 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.186585 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.186586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.186587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.186689 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.186763 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.186920 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.186986 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:29:43.187085 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.181630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.13.15.131:40333)
0929 06:29:43.181752 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.181761 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.181817 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.181931 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.181932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.182085 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:43.182171 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.182173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.182748 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.182748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.183256 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.183259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.183616 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.183617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.183673 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.184078 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:43.184082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:43.184571 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:43.185997 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.185998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.185998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.186224 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.186460 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.186461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.186461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.186516 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.186756 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:43.186756 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:43.186809 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.186893 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.186988 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260929 06:29:43.190507 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 06:29:43.210666 20972 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:43.205431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.13.15.190:37973)
0929 06:29:43.205496 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.205498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.205507 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:43.205537 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.205538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.205545 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.205607 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.206158 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.206658 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.207408 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.207613 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.207692 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.210558 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43820: BlockingRecv error: recv got EOF from 127.0.0.1:43820 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:43.217483 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.211318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.13.15.190:37973)
0929 06:29:43.211371 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.211374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.211456 (+ 82us) server_negotiation.cc:407] Connection header received
0929 06:29:43.211514 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.211515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.211523 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.211574 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.212082 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.212532 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.213196 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.213374 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.213454 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.215445 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.215447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.215447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.215452 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.216578 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.216875 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.216876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.216876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.216919 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.217092 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.217092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.217093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.217198 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.217280 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.217355 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.217404 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:29:43.219063 20923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:29:43.220588 20923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:43.227150 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.222773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:44383 (local address 127.0.0.1:37506)
0929 06:29:43.222867 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.222906 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.222961 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.223553 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.223554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.223556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.223616 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.223618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224810 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.224811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.225857 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.225859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226897 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.226970 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.226974 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.227005 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.227049 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":35,"thread_start_us":54,"threads_started":1}
I20260929 06:29:43.227164 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.222792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:32781 (local address 127.0.0.1:57286)
0929 06:29:43.223076 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.223087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.223137 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.223328 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.223331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.223333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.223395 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.223397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224770 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.224770 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.226046 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.226048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226967 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.227019 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.227022 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.227063 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.227098 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":45,"thread_start_us":78,"threads_started":1}
I20260929 06:29:43.227208 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.223003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.13.15.130:32781)
0929 06:29:43.223150 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.223156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.223167 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:29:43.223205 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.223208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.223224 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.223304 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.223487 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224658 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226127 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226905 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226968 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.227040 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.227071 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.227106 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260929 06:29:43.227555 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.222867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.13.15.131:44383)
0929 06:29:43.223338 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.223343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.223352 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:43.223386 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.223388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.223403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.223482 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.223704 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224749 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.225937 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226769 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.227366 (+ 597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.227427 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.227447 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.227478 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":78,"threads_started":1}
I20260929 06:29:43.228032 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.222960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:42045 (local address 127.0.0.1:38168)
0929 06:29:43.223156 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.223166 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.223205 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.223780 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.223781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.223783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.223827 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.223828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224704 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.224705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.226051 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.226052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226812 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.226888 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.226902 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.226967 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.227018 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":26,"thread_start_us":76,"threads_started":1}
I20260929 06:29:43.227980 21200 tablet_service.cc:1467] Processing CreateTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef (DEFAULT_TABLE table=test-table [id=0d1e40af76ed4363b88e3b644426c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.228169 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.223172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38168 (local address 127.13.15.129:42045)
0929 06:29:43.223612 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.223616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.223626 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:29:43.223666 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.223668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.223682 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.223897 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.223939 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.224638 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226185 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.226711 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.226801 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.226882 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.227095 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.227139 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":97,"threads_started":1}
I20260929 06:29:43.228359 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad72d4a4d49c4308b7c5a5180642f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.228269 21340 tablet_service.cc:1467] Processing CreateTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef (DEFAULT_TABLE table=test-table [id=0d1e40af76ed4363b88e3b644426c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.228614 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad72d4a4d49c4308b7c5a5180642f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.230955 21060 tablet_service.cc:1467] Processing CreateTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef (DEFAULT_TABLE table=test-table [id=0d1e40af76ed4363b88e3b644426c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.231261 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad72d4a4d49c4308b7c5a5180642f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.231460 21434 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Bootstrap starting.
I20260929 06:29:43.231740 21435 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Bootstrap starting.
I20260929 06:29:43.232302 21435 tablet_bootstrap.cc:654] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.232314 21434 tablet_bootstrap.cc:654] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.232642 21435 log.cc:824] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:43.232642 21434 log.cc:824] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:43.233414 21435 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: No bootstrap required, opened a new log
I20260929 06:29:43.233493 21435 ts_tablet_manager.cc:1397] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.233671 21434 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: No bootstrap required, opened a new log
I20260929 06:29:43.233739 21434 ts_tablet_manager.cc:1397] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.234663 21438 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Bootstrap starting.
I20260929 06:29:43.235294 21438 tablet_bootstrap.cc:654] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.235538 21434 raft_consensus.cc:348] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.235606 21438 log.cc:824] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:43.235677 21434 raft_consensus.cc:374] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.235584 21435 raft_consensus.cc:348] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.235720 21434 raft_consensus.cc:729] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb76ad97af27428ba2cd775a48487b72, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.235735 21435 raft_consensus.cc:374] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.235776 21435 raft_consensus.cc:729] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf593401a64172976745e95e780011, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.235826 21434 consensus_queue.cc:260] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.235890 21435 consensus_queue.cc:260] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.236085 21434 ts_tablet_manager.cc:1428] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 06:29:43.236142 21435 ts_tablet_manager.cc:1428] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:29:43.236239 21406 heartbeater.cc:503] Master 127.13.15.190:37973 was elected leader, sending a full tablet report...
I20260929 06:29:43.236249 21266 heartbeater.cc:503] Master 127.13.15.190:37973 was elected leader, sending a full tablet report...
I20260929 06:29:43.236372 21438 tablet_bootstrap.cc:492] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: No bootstrap required, opened a new log
I20260929 06:29:43.236449 21438 ts_tablet_manager.cc:1397] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.238076 21438 raft_consensus.cc:348] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.238233 21438 raft_consensus.cc:374] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.238281 21438 raft_consensus.cc:729] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.238394 21438 consensus_queue.cc:260] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.238610 21438 ts_tablet_manager.cc:1428] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:29:43.238678 21126 heartbeater.cc:503] Master 127.13.15.190:37973 was elected leader, sending a full tablet report...
W20260929 06:29:43.271495 21267 tablet.cc:2369] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:43.359231 21442 raft_consensus.cc:482] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:43.359352 21442 raft_consensus.cc:504] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.359781 21442 leader_election.cc:288] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbf593401a64172976745e95e780011 (127.13.15.131:44383), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:43.362895 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.360043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:57969 (local address 127.13.15.130:32781)
0929 06:29:43.360093 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.360095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.360108 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:29:43.360186 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.360188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.360198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.360288 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.360452 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.361051 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362003 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.362532 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362588 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.362595 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.362786 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.362828 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:29:43.362910 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.359981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:32781 (local address 127.13.15.129:57969)
0929 06:29:43.360042 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.360054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.360100 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.360311 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.360312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.360314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.360381 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.360383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.361125 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.361126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.361917 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.361919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.362618 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.362708 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.362711 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.362764 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.362813 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 06:29:43.362941 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.359982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:44383 (local address 127.13.15.129:34531)
0929 06:29:43.360226 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.360236 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.360310 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.360453 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.360454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.360455 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.360527 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.360528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.361288 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.361289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.362023 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.362025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.362684 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.362742 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.362745 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.362820 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.362853 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":90,"thread_start_us":98,"threads_started":1}
I20260929 06:29:43.362943 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.360061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:34531 (local address 127.13.15.131:44383)
0929 06:29:43.360105 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.360107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.360311 (+ 204us) server_negotiation.cc:407] Connection header received
0929 06:29:43.360373 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.360374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.360383 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.360438 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.360598 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.361212 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362122 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.362604 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.362675 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.362683 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.362836 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.362870 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:29:43.363276 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad72d4a4d49c4308b7c5a5180642f0ef" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72" is_pre_election: true
I20260929 06:29:43.363265 21360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad72d4a4d49c4308b7c5a5180642f0ef" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbf593401a64172976745e95e780011" is_pre_election: true
I20260929 06:29:43.363409 21220 raft_consensus.cc:2454] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 0.
I20260929 06:29:43.363409 21360 raft_consensus.cc:2454] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 0.
I20260929 06:29:43.363616 21014 leader_election.cc:302] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, 4dbf593401a64172976745e95e780011; no voters:
I20260929 06:29:43.363731 21442 raft_consensus.cc:2781] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:43.363802 21442 raft_consensus.cc:482] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:43.363835 21442 raft_consensus.cc:3037] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.364511 21442 raft_consensus.cc:504] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.364670 21442 leader_election.cc:288] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4dbf593401a64172976745e95e780011 (127.13.15.131:44383), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:43.364821 21360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad72d4a4d49c4308b7c5a5180642f0ef" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbf593401a64172976745e95e780011"
I20260929 06:29:43.364943 21360 raft_consensus.cc:3037] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Advancing to term 1
W20260929 06:29:43.365098 21127 tablet.cc:2369] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:43.365330 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad72d4a4d49c4308b7c5a5180642f0ef" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72"
I20260929 06:29:43.365391 21220 raft_consensus.cc:3037] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.365935 21360 raft_consensus.cc:2454] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 1.
I20260929 06:29:43.366060 21220 raft_consensus.cc:2454] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 1.
I20260929 06:29:43.366189 21014 leader_election.cc:302] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, 4dbf593401a64172976745e95e780011; no voters:
I20260929 06:29:43.366307 21442 raft_consensus.cc:2781] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:43.366535 21442 raft_consensus.cc:686] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Running, Role: LEADER
I20260929 06:29:43.366641 21442 consensus_queue.cc:237] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.367511 20923 catalog_manager.cc:5495] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "2dc526a0efba4786a173766eaa1227d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } health_report { overall_health: HEALTHY } } }
I20260929 06:29:43.373272 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.371200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.13.15.129:42045)
0929 06:29:43.371263 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.371265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.371274 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:43.371298 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.371298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.371311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.371367 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:43.371480 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.371910 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.372596 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.372768 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.372820 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.372948 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.373040 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.373101 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.373209 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.373223 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 06:29:43.375171 21360 raft_consensus.cc:1261] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Refusing update from remote peer 2dc526a0efba4786a173766eaa1227d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.375458 21220 raft_consensus.cc:1261] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Refusing update from remote peer 2dc526a0efba4786a173766eaa1227d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.375483 21442 consensus_queue.cc:1035] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.376782 21442 consensus_queue.cc:1035] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.379105 21451 mvcc.cc:205] Tried to move back new op lower bound from 7334557218302201856 to 7334557218270449664. Current Snapshot: MvccSnapshot[applied={T|T < 7334557218302201856}]
I20260929 06:29:43.380299 20922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.380393 20922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.381116 20922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Sending DeleteTablet for 3 replicas of tablet ad72d4a4d49c4308b7c5a5180642f0ef
I20260929 06:29:43.381449 21340 tablet_service.cc:1514] Processing DeleteTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37506
I20260929 06:29:43.381577 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57286
I20260929 06:29:43.381757 21457 tablet_replica.cc:330] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: stopping tablet replica
I20260929 06:29:43.381798 21458 tablet_replica.cc:330] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: stopping tablet replica
I20260929 06:29:43.381894 21458 raft_consensus.cc:2229] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.381924 21457 raft_consensus.cc:2229] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.381994 21458 raft_consensus.cc:2258] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.381981 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet ad72d4a4d49c4308b7c5a5180642f0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260929 06:29:43.382182 21457 raft_consensus.cc:2258] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.382298 20922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 06:29:43.382444 21458 ts_tablet_manager.cc:1905] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 06:29:43.382452 20922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:43.382853 21457 ts_tablet_manager.cc:1905] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.384054 21340 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 (DEFAULT_TABLE table=test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.384187 21458 ts_tablet_manager.cc:1918] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.384215 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee922bc39c3422c9be81a9cd0fd3973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.384270 21458 log.cc:1196] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/ad72d4a4d49c4308b7c5a5180642f0ef
I20260929 06:29:43.384404 21060 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 (DEFAULT_TABLE table=test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.384526 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee922bc39c3422c9be81a9cd0fd3973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.384534 21458 ts_tablet_manager.cc:1939] T ad72d4a4d49c4308b7c5a5180642f0ef P cb76ad97af27428ba2cd775a48487b72: Deleting consensus metadata
I20260929 06:29:43.384886 20908 catalog_manager.cc:4841] TS cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781): tablet ad72d4a4d49c4308b7c5a5180642f0ef (table test-table [id=0d1e40af76ed4363b88e3b644426c6ca]) successfully deleted
I20260929 06:29:43.385192 21457 ts_tablet_manager.cc:1918] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.385313 21457 log.cc:1196] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/ad72d4a4d49c4308b7c5a5180642f0ef
I20260929 06:29:43.385607 21457 ts_tablet_manager.cc:1939] T ad72d4a4d49c4308b7c5a5180642f0ef P 4dbf593401a64172976745e95e780011: Deleting consensus metadata
I20260929 06:29:43.385948 21438 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Bootstrap starting.
I20260929 06:29:43.386025 20910 catalog_manager.cc:4841] TS 4dbf593401a64172976745e95e780011 (127.13.15.131:44383): tablet ad72d4a4d49c4308b7c5a5180642f0ef (table test-table [id=0d1e40af76ed4363b88e3b644426c6ca]) successfully deleted
I20260929 06:29:43.386379 21438 tablet_bootstrap.cc:654] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.387043 21438 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: No bootstrap required, opened a new log
I20260929 06:29:43.387094 21438 ts_tablet_manager.cc:1397] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:43.387172 21435 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Bootstrap starting.
I20260929 06:29:43.387219 21438 raft_consensus.cc:348] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.387291 21438 raft_consensus.cc:374] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.387311 21438 raft_consensus.cc:729] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.387334 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 (DEFAULT_TABLE table=test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.387354 21438 consensus_queue.cc:260] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.387431 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee922bc39c3422c9be81a9cd0fd3973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.387431 21438 ts_tablet_manager.cc:1428] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:43.387560 21435 tablet_bootstrap.cc:654] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.388461 21434 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Bootstrap starting.
I20260929 06:29:43.388851 21459 tablet_replica.cc:330] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: stopping tablet replica
I20260929 06:29:43.388877 21434 tablet_bootstrap.cc:654] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.388916 21435 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: No bootstrap required, opened a new log
I20260929 06:29:43.388944 21459 raft_consensus.cc:2229] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:43.388952 21435 ts_tablet_manager.cc:1397] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.389046 21459 raft_consensus.cc:2258] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.389084 21435 raft_consensus.cc:348] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.389151 21435 raft_consensus.cc:374] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.389176 21435 raft_consensus.cc:729] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf593401a64172976745e95e780011, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.389217 21435 consensus_queue.cc:260] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.389304 21435 ts_tablet_manager.cc:1428] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.389395 21434 tablet_bootstrap.cc:492] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: No bootstrap required, opened a new log
I20260929 06:29:43.389431 21434 ts_tablet_manager.cc:1397] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:43.389564 21434 raft_consensus.cc:348] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.389632 21434 raft_consensus.cc:374] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.389653 21434 raft_consensus.cc:729] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb76ad97af27428ba2cd775a48487b72, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.389690 21434 consensus_queue.cc:260] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.389698 21459 ts_tablet_manager.cc:1905] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.389766 21434 ts_tablet_manager.cc:1428] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.391419 21459 ts_tablet_manager.cc:1918] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.391491 21459 log.cc:1196] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/ad72d4a4d49c4308b7c5a5180642f0ef
I20260929 06:29:43.391799 21459 ts_tablet_manager.cc:1939] T ad72d4a4d49c4308b7c5a5180642f0ef P 2dc526a0efba4786a173766eaa1227d0: Deleting consensus metadata
I20260929 06:29:43.392194 20910 catalog_manager.cc:4841] TS 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045): tablet ad72d4a4d49c4308b7c5a5180642f0ef (table test-table [id=0d1e40af76ed4363b88e3b644426c6ca]) successfully deleted
I20260929 06:29:43.423650 21441 raft_consensus.cc:482] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:43.423779 21441 raft_consensus.cc:504] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.424197 21441 leader_election.cc:288] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
W20260929 06:29:43.427776 21407 tablet.cc:2369] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:29:43.428074 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.424368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:42045 (local address 127.13.15.131:33417)
0929 06:29:43.424479 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.424511 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.424564 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.424739 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.424740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.424742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.424809 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.424810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.425727 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.425728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.427017 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.427020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.427807 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.427808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.427893 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.427896 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.427944 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.427984 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 06:29:43.428375 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.424526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:33417 (local address 127.13.15.129:42045)
0929 06:29:43.424583 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.424585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.424594 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:43.424622 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.424623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.424633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.424689 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.424875 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.425565 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.427129 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.427694 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.427767 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.427776 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.428271 (+ 495us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.428312 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 06:29:43.428670 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.424899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:36929 (local address 127.13.15.130:32781)
0929 06:29:43.424969 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.424971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.424979 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:43.425002 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.425003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.425012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.425080 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:29:43.425306 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.426036 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.427418 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.427942 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.428049 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.428060 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.428539 (+ 479us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.428588 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 06:29:43.428625 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee922bc39c3422c9be81a9cd0fd3973" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0" is_pre_election: true
I20260929 06:29:43.428762 21077 raft_consensus.cc:2454] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:43.428972 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.424390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:32781 (local address 127.13.15.131:36929)
0929 06:29:43.424687 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.424699 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.424748 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.425119 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.425120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.425122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:29:43.425218 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.425220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.426165 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.426166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.427318 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.427320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.428328 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.428329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.428390 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.428394 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.428826 (+ 432us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.428874 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":82,"threads_started":1}
I20260929 06:29:43.428948 21294 leader_election.cc:302] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, 4dbf593401a64172976745e95e780011; no voters:
I20260929 06:29:43.429131 21441 raft_consensus.cc:2781] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:43.429169 21441 raft_consensus.cc:482] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:43.429191 21441 raft_consensus.cc:3037] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.429301 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee922bc39c3422c9be81a9cd0fd3973" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72" is_pre_election: true
I20260929 06:29:43.429426 21220 raft_consensus.cc:2454] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:43.430044 21441 raft_consensus.cc:504] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.430207 21441 leader_election.cc:288] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Requested vote from peers 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:43.430356 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee922bc39c3422c9be81a9cd0fd3973" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0"
I20260929 06:29:43.430428 21077 raft_consensus.cc:3037] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.430437 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee922bc39c3422c9be81a9cd0fd3973" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72"
I20260929 06:29:43.430521 21220 raft_consensus.cc:3037] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.431188 21220 raft_consensus.cc:2454] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:43.431214 21077 raft_consensus.cc:2454] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:43.431361 21292 leader_election.cc:302] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbf593401a64172976745e95e780011, cb76ad97af27428ba2cd775a48487b72; no voters:
I20260929 06:29:43.431466 21441 raft_consensus.cc:2781] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:43.431627 21441 raft_consensus.cc:686] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbf593401a64172976745e95e780011, State: Running, Role: LEADER
I20260929 06:29:43.431685 21441 consensus_queue.cc:237] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.432395 20922 catalog_manager.cc:5495] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbf593401a64172976745e95e780011 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } health_report { overall_health: HEALTHY } } }
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663382, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.190@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:43.458631 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.450936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.13.15.190:37973)
0929 06:29:43.450983 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.450985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.451133 (+ 148us) server_negotiation.cc:407] Connection header received
0929 06:29:43.451164 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.451165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.451174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.451233 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.451798 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.452286 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.453300 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.453482 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.453564 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.455970 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.455971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.455971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.455977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.457417 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.457832 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.457833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.457833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.457904 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.458113 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.458114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.458114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.458271 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.458322 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.458502 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.458567 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:29:43.458766 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.450843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:37973 (local address 127.0.0.1:43848)
0929 06:29:43.451019 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.451031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.451115 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.451266 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.451267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.451657 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:43.451713 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.451716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.452372 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.452373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.453185 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.453187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.453574 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.453575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.453636 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.454229 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:43.454234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:43.455873 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:43.457502 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.457508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.457510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.457754 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.457975 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:43.457976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:43.457976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:43.458049 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:43.458337 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:43.458343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:43.458400 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.458480 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.458640 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":48,"thread_start_us":87,"threads_started":1}
I20260929 06:29:43.460945 20922 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:43848:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 06:29:43.461166 20922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:43.462558 21200 tablet_service.cc:1467] Processing CreateTablet for tablet a65d408dc77447a6b5a39dea04d8a4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20e92d7d17dd424694765d0d3de114ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:29:43.462626 21340 tablet_service.cc:1467] Processing CreateTablet for tablet a65d408dc77447a6b5a39dea04d8a4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20e92d7d17dd424694765d0d3de114ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:29:43.462649 21060 tablet_service.cc:1467] Processing CreateTablet for tablet a65d408dc77447a6b5a39dea04d8a4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20e92d7d17dd424694765d0d3de114ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 06:29:43.462713 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65d408dc77447a6b5a39dea04d8a4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.462778 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65d408dc77447a6b5a39dea04d8a4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.462766 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65d408dc77447a6b5a39dea04d8a4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.464212 21435 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: Bootstrap starting.
I20260929 06:29:43.464289 21434 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: Bootstrap starting.
I20260929 06:29:43.464586 21435 tablet_bootstrap.cc:654] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.464699 21434 tablet_bootstrap.cc:654] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.464907 21438 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: Bootstrap starting.
I20260929 06:29:43.465317 21438 tablet_bootstrap.cc:654] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.465912 21435 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: No bootstrap required, opened a new log
I20260929 06:29:43.465932 21438 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: No bootstrap required, opened a new log
I20260929 06:29:43.465979 21438 ts_tablet_manager.cc:1397] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:43.465982 21435 ts_tablet_manager.cc:1397] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.466128 21438 raft_consensus.cc:348] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.466121 21435 raft_consensus.cc:348] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.466228 21435 raft_consensus.cc:374] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.466228 21438 raft_consensus.cc:374] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.466250 21435 raft_consensus.cc:729] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf593401a64172976745e95e780011, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.466260 21438 raft_consensus.cc:729] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.466301 21438 consensus_queue.cc:260] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.466325 21435 consensus_queue.cc:260] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.466418 21438 ts_tablet_manager.cc:1428] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.466426 21435 ts_tablet_manager.cc:1428] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.466466 21441 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.466536 21441 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.466531 21454 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.466606 21454 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.467219 21434 tablet_bootstrap.cc:492] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: No bootstrap required, opened a new log
I20260929 06:29:43.467280 21434 ts_tablet_manager.cc:1397] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 06:29:43.467433 21434 raft_consensus.cc:348] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.467494 21434 raft_consensus.cc:374] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.467514 21434 raft_consensus.cc:729] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb76ad97af27428ba2cd775a48487b72, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.467561 21434 consensus_queue.cc:260] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.467644 21434 ts_tablet_manager.cc:1428] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.467768 21440 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.467847 21440 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.504624 21441 raft_consensus.cc:482] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:43.504765 21441 raft_consensus.cc:504] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.504941 21441 leader_election.cc:288] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781), 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045)
I20260929 06:29:43.505177 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65d408dc77447a6b5a39dea04d8a4cf" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72" is_pre_election: true
I20260929 06:29:43.505174 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65d408dc77447a6b5a39dea04d8a4cf" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0" is_pre_election: true
I20260929 06:29:43.505290 21220 raft_consensus.cc:2454] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:43.505286 21077 raft_consensus.cc:2454] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:43.505458 21294 leader_election.cc:302] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, 4dbf593401a64172976745e95e780011; no voters:
I20260929 06:29:43.505573 21441 raft_consensus.cc:2781] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:43.505622 21441 raft_consensus.cc:482] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:43.505653 21441 raft_consensus.cc:3037] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.506590 21441 raft_consensus.cc:504] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.506750 21441 leader_election.cc:288] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Requested vote from peers cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781), 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045)
I20260929 06:29:43.506924 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65d408dc77447a6b5a39dea04d8a4cf" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0"
I20260929 06:29:43.506999 21077 raft_consensus.cc:3037] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.506953 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65d408dc77447a6b5a39dea04d8a4cf" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72"
I20260929 06:29:43.507040 21220 raft_consensus.cc:3037] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.507596 21220 raft_consensus.cc:2454] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:43.507649 21077 raft_consensus.cc:2454] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:43.507790 21292 leader_election.cc:302] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbf593401a64172976745e95e780011, cb76ad97af27428ba2cd775a48487b72; no voters:
I20260929 06:29:43.507891 21441 raft_consensus.cc:2781] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:43.507953 21441 raft_consensus.cc:686] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbf593401a64172976745e95e780011, State: Running, Role: LEADER
I20260929 06:29:43.508008 21441 consensus_queue.cc:237] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } }
I20260929 06:29:43.508322 21464 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: TxnStatusTablet state changed. Reason: New leader 4dbf593401a64172976745e95e780011. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.508419 21464 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:29:43.508672 21476 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 06:29:43.508729 20923 catalog_manager.cc:5495] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbf593401a64172976745e95e780011 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } health_report { overall_health: UNKNOWN } } }
I20260929 06:29:43.508816 21476 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 06:29:43.514819 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.512011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.13.15.131:44383)
0929 06:29:43.512052 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.512054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.512131 (+ 77us) server_negotiation.cc:407] Connection header received
0929 06:29:43.512227 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.512228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.512242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.512293 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:43.512422 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.512918 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.514025 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.514269 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.514324 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.514531 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.514619 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.514667 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.514752 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.514770 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:29:43.514873 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.511967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:44383 (local address 127.0.0.1:37510)
0929 06:29:43.512079 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.512092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.512151 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.512299 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.512300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.512301 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:29:43.512365 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.512368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.512990 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.512991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.513938 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.513940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.514340 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.514341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.514420 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.514436 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.514684 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.514692 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.514750 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.514795 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 06:29:43.515980 21077 raft_consensus.cc:1261] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.516202 21220 raft_consensus.cc:1261] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.516280 21464 consensus_queue.cc:1035] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.516409 21441 consensus_queue.cc:1035] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.516923 21440 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: TxnStatusTablet state changed. Reason: New leader 4dbf593401a64172976745e95e780011. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.517012 21440 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.517077 21454 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: TxnStatusTablet state changed. Reason: New leader 4dbf593401a64172976745e95e780011. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.517175 21454 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.517679 21464 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.517758 21464 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:29:43.517849 21448 mvcc.cc:205] Tried to move back new op lower bound from 7334557218879344640 to 7334557218849103872. Current Snapshot: MvccSnapshot[applied={T|T < 7334557218879344640 or (T in {7334557218879344640})}]
I20260929 06:29:43.518525 21440 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.518604 21440 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P cb76ad97af27428ba2cd775a48487b72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.519407 21441 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.519505 21441 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 4dbf593401a64172976745e95e780011: This TxnStatusTablet replica's current role is: LEADER
I20260929 06:29:43.519977 21447 mvcc.cc:205] Tried to move back new op lower bound from 7334557218879344640 to 7334557218849103872. Current Snapshot: MvccSnapshot[applied={T|T < 7334557218879344640}]
I20260929 06:29:43.520076 21454 tablet_replica.cc:439] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } } peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } }
I20260929 06:29:43.520147 21454 tablet_replica.cc:442] T a65d408dc77447a6b5a39dea04d8a4cf P 2dc526a0efba4786a173766eaa1227d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 06:29:43.522156 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.519348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.13.15.131:44383)
0929 06:29:43.519511 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.519512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.519519 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:43.519538 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.519539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.519566 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.519609 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:43.519745 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.520430 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.521479 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.521651 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.521726 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.521887 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.521921 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.521973 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.522046 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.522066 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260929 06:29:43.526758 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.524437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:44383 (local address 127.13.15.131:33545)
0929 06:29:43.524504 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:43.524515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:43.524557 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:43.524699 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:43.524700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:43.524702 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 06:29:43.524748 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.524749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.525351 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.525352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.525920 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:43.525921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.526325 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.526326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:43.526397 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.526411 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.526581 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.526582 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 06:29:43.526628 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:29:43.526665 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":3}
I20260929 06:29:43.526758 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.524498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:33545 (local address 127.13.15.131:44383)
0929 06:29:43.524532 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.524534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.524543 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:43.524603 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.524604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.524613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.524678 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:43.524844 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.525263 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.526004 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.526243 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.526324 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.526493 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.526521 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.526572 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.526629 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.526647 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:29:43.528699 21077 raft_consensus.cc:1261] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.528861 21220 raft_consensus.cc:1261] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.529022 21483 consensus_queue.cc:1035] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.529124 21464 consensus_queue.cc:1035] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.540050 20922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.540144 20922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.540661 20922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Sending DeleteTablet for 3 replicas of tablet 6ee922bc39c3422c9be81a9cd0fd3973
I20260929 06:29:43.540890 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57286
I20260929 06:29:43.540947 21339 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37506
I20260929 06:29:43.540977 21458 tablet_replica.cc:330] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: stopping tablet replica
I20260929 06:29:43.541019 21458 raft_consensus.cc:2229] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.541049 21458 raft_consensus.cc:2258] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.541066 21457 tablet_replica.cc:330] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: stopping tablet replica
I20260929 06:29:43.541113 21457 raft_consensus.cc:2229] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:43.541203 21457 raft_consensus.cc:2258] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.541388 21458 ts_tablet_manager.cc:1905] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.541522 21457 ts_tablet_manager.cc:1905] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.543079 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee922bc39c3422c9be81a9cd0fd3973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260929 06:29:43.543205 21459 tablet_replica.cc:330] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: stopping tablet replica
I20260929 06:29:43.543272 21459 raft_consensus.cc:2229] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.543306 21459 raft_consensus.cc:2258] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.543308 21457 ts_tablet_manager.cc:1918] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:29:43.543354 21457 log.cc:1196] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/6ee922bc39c3422c9be81a9cd0fd3973
I20260929 06:29:43.543576 21457 ts_tablet_manager.cc:1939] T 6ee922bc39c3422c9be81a9cd0fd3973 P 4dbf593401a64172976745e95e780011: Deleting consensus metadata
I20260929 06:29:43.543851 21458 ts_tablet_manager.cc:1918] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:29:43.543886 20910 catalog_manager.cc:4841] TS 4dbf593401a64172976745e95e780011 (127.13.15.131:44383): tablet 6ee922bc39c3422c9be81a9cd0fd3973 (table test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]) successfully deleted
I20260929 06:29:43.543915 21458 log.cc:1196] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/6ee922bc39c3422c9be81a9cd0fd3973
I20260929 06:29:43.544103 21458 ts_tablet_manager.cc:1939] T 6ee922bc39c3422c9be81a9cd0fd3973 P cb76ad97af27428ba2cd775a48487b72: Deleting consensus metadata
I20260929 06:29:43.544382 20908 catalog_manager.cc:4841] TS cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781): tablet 6ee922bc39c3422c9be81a9cd0fd3973 (table test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]) successfully deleted
I20260929 06:29:43.545181 21459 ts_tablet_manager.cc:1905] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.546682 21459 ts_tablet_manager.cc:1918] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:29:43.546752 21459 log.cc:1196] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/6ee922bc39c3422c9be81a9cd0fd3973
I20260929 06:29:43.546964 21459 ts_tablet_manager.cc:1939] T 6ee922bc39c3422c9be81a9cd0fd3973 P 2dc526a0efba4786a173766eaa1227d0: Deleting consensus metadata
I20260929 06:29:43.547225 20910 catalog_manager.cc:4841] TS 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045): tablet 6ee922bc39c3422c9be81a9cd0fd3973 (table test-table [id=23fbdb171eee4d1898fdd4b84c3265b6]) successfully deleted
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 06:29:43.565466 20972 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:43.559875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.13.15.190:37973)
0929 06:29:43.559935 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.559938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.560040 (+ 102us) server_negotiation.cc:407] Connection header received
0929 06:29:43.560106 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.560107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.560118 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.560172 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.560669 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.561466 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.562239 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.562545 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.562621 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.565396 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43856: BlockingRecv error: recv got EOF from 127.0.0.1:43856 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 06:29:43 dist-test-slave-sm0p krb5kdc[20878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM
I20260929 06:29:43.573557 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.566223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.13.15.190:37973)
0929 06:29:43.566259 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.566261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.566438 (+ 177us) server_negotiation.cc:407] Connection header received
0929 06:29:43.566494 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.566494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.566502 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.566542 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:43.567154 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.567636 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.568613 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.568811 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.568873 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.571171 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:43.571172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:43.571173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:43.571180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:43.572459 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.572791 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.572792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.572792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.572850 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:43.573063 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:43.573064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:43.573064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:43.573174 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:43.573275 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.573386 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.573454 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:29:43.575241 20922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:29:43.575402 20922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:43.576781 21060 tablet_service.cc:1467] Processing CreateTablet for tablet 117103bbaf464932948f09263808cacc (DEFAULT_TABLE table=test-table [id=5b57784586e046a7aa16a2de44494ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.576881 21339 tablet_service.cc:1467] Processing CreateTablet for tablet 117103bbaf464932948f09263808cacc (DEFAULT_TABLE table=test-table [id=5b57784586e046a7aa16a2de44494ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.576921 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117103bbaf464932948f09263808cacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.576900 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 117103bbaf464932948f09263808cacc (DEFAULT_TABLE table=test-table [id=5b57784586e046a7aa16a2de44494ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.576997 21339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117103bbaf464932948f09263808cacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.577013 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117103bbaf464932948f09263808cacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.578415 21435 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Bootstrap starting.
I20260929 06:29:43.578503 21438 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Bootstrap starting.
I20260929 06:29:43.578816 21435 tablet_bootstrap.cc:654] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.578831 21438 tablet_bootstrap.cc:654] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.579119 21434 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Bootstrap starting.
I20260929 06:29:43.579365 21438 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: No bootstrap required, opened a new log
I20260929 06:29:43.579403 21438 ts_tablet_manager.cc:1397] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 06:29:43.579486 21434 tablet_bootstrap.cc:654] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.579500 21438 raft_consensus.cc:348] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.579552 21438 raft_consensus.cc:374] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.579567 21438 raft_consensus.cc:729] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.579641 21438 consensus_queue.cc:260] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.579737 21438 ts_tablet_manager.cc:1428] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.580581 21434 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: No bootstrap required, opened a new log
I20260929 06:29:43.580634 21434 ts_tablet_manager.cc:1397] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 06:29:43.580775 21434 raft_consensus.cc:348] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.580844 21434 raft_consensus.cc:374] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.580864 21434 raft_consensus.cc:729] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb76ad97af27428ba2cd775a48487b72, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.580902 21434 consensus_queue.cc:260] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.580982 21434 ts_tablet_manager.cc:1428] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:43.581243 21435 tablet_bootstrap.cc:492] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: No bootstrap required, opened a new log
I20260929 06:29:43.581293 21435 ts_tablet_manager.cc:1397] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 06:29:43.581418 21435 raft_consensus.cc:348] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.581483 21435 raft_consensus.cc:374] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.581503 21435 raft_consensus.cc:729] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf593401a64172976745e95e780011, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.581542 21435 consensus_queue.cc:260] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.581625 21435 ts_tablet_manager.cc:1428] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.679095 21454 raft_consensus.cc:482] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:43.679236 21454 raft_consensus.cc:504] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.679459 21454 leader_election.cc:288] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781), 4dbf593401a64172976745e95e780011 (127.13.15.131:44383)
I20260929 06:29:43.679672 21360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117103bbaf464932948f09263808cacc" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbf593401a64172976745e95e780011" is_pre_election: true
I20260929 06:29:43.679703 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117103bbaf464932948f09263808cacc" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72" is_pre_election: true
I20260929 06:29:43.679766 21360 raft_consensus.cc:2454] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 0.
I20260929 06:29:43.679781 21220 raft_consensus.cc:2454] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 0.
I20260929 06:29:43.679952 21012 leader_election.cc:302] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, cb76ad97af27428ba2cd775a48487b72; no voters:
I20260929 06:29:43.680047 21454 raft_consensus.cc:2781] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:43.680073 21454 raft_consensus.cc:482] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:43.680092 21454 raft_consensus.cc:3037] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.680926 21454 raft_consensus.cc:504] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.681071 21454 leader_election.cc:288] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 election: Requested vote from peers cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781), 4dbf593401a64172976745e95e780011 (127.13.15.131:44383)
I20260929 06:29:43.681221 21360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117103bbaf464932948f09263808cacc" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbf593401a64172976745e95e780011"
I20260929 06:29:43.681231 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117103bbaf464932948f09263808cacc" candidate_uuid: "2dc526a0efba4786a173766eaa1227d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72"
I20260929 06:29:43.681300 21360 raft_consensus.cc:3037] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.681309 21220 raft_consensus.cc:3037] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:43.682008 21220 raft_consensus.cc:2454] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 1.
I20260929 06:29:43.682021 21360 raft_consensus.cc:2454] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dc526a0efba4786a173766eaa1227d0 in term 1.
I20260929 06:29:43.682199 21012 leader_election.cc:302] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, cb76ad97af27428ba2cd775a48487b72; no voters:
I20260929 06:29:43.682291 21454 raft_consensus.cc:2781] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:43.682336 21454 raft_consensus.cc:686] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 1 LEADER]: Becoming Leader. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Running, Role: LEADER
I20260929 06:29:43.682389 21454 consensus_queue.cc:237] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.683015 20922 catalog_manager.cc:5495] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "2dc526a0efba4786a173766eaa1227d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } health_report { overall_health: UNKNOWN } } }
I20260929 06:29:43.699419 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:43.696880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.13.15.129:42045)
0929 06:29:43.696920 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:43.696921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:43.696952 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:29:43.697023 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:43.697024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:43.697032 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:43.697078 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:43.697227 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.697665 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.698763 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:43.698947 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:43.698999 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:43.699163 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:43.699191 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:43.699232 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:43.699315 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 06:29:43.699333 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:29:43.700222 21360 raft_consensus.cc:1261] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Refusing update from remote peer 2dc526a0efba4786a173766eaa1227d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.700392 21220 raft_consensus.cc:1261] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Refusing update from remote peer 2dc526a0efba4786a173766eaa1227d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:43.700495 21454 consensus_queue.cc:1035] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.700589 21442 consensus_queue.cc:1035] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:43.703011 20923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.703104 20923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:43.703896 20923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Sending DeleteTablet for 3 replicas of tablet 117103bbaf464932948f09263808cacc
I20260929 06:29:43.704141 21339 tablet_service.cc:1514] Processing DeleteTablet for tablet 117103bbaf464932948f09263808cacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37506
I20260929 06:29:43.704228 21457 tablet_replica.cc:330] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: stopping tablet replica
I20260929 06:29:43.704192 21059 tablet_service.cc:1514] Processing DeleteTablet for tablet 117103bbaf464932948f09263808cacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260929 06:29:43.704223 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 117103bbaf464932948f09263808cacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57286
I20260929 06:29:43.704270 21457 raft_consensus.cc:2229] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.704295 21458 tablet_replica.cc:330] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: stopping tablet replica
I20260929 06:29:43.704306 21457 raft_consensus.cc:2258] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.704334 21458 raft_consensus.cc:2229] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:43.704377 21458 raft_consensus.cc:2258] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.704375 21459 tablet_replica.cc:330] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: stopping tablet replica
I20260929 06:29:43.704424 21459 raft_consensus.cc:2229] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:43.704196 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet 117103bbaf464932948f09263808cacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260929 06:29:43.704614 21339 tablet_service.cc:1514] Processing DeleteTablet for tablet 117103bbaf464932948f09263808cacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:43 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37506
I20260929 06:29:43.704469 21459 raft_consensus.cc:2258] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:43.705123 21457 ts_tablet_manager.cc:1905] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.705329 21459 ts_tablet_manager.cc:1905] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.705343 21458 ts_tablet_manager.cc:1905] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:43.706665 21457 ts_tablet_manager.cc:1918] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.706720 21457 log.cc:1196] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/117103bbaf464932948f09263808cacc
I20260929 06:29:43.706924 21457 ts_tablet_manager.cc:1939] T 117103bbaf464932948f09263808cacc P 4dbf593401a64172976745e95e780011: Deleting consensus metadata
I20260929 06:29:43.706943 21458 ts_tablet_manager.cc:1918] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.706804 20923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 06:29:43.706991 21458 log.cc:1196] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/117103bbaf464932948f09263808cacc
W20260929 06:29:43.707023 20923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:29:43.707214 21458 ts_tablet_manager.cc:1939] T 117103bbaf464932948f09263808cacc P cb76ad97af27428ba2cd775a48487b72: Deleting consensus metadata
I20260929 06:29:43.707522 20910 catalog_manager.cc:4841] TS 4dbf593401a64172976745e95e780011 (127.13.15.131:44383): tablet 117103bbaf464932948f09263808cacc (table test-table [id=5b57784586e046a7aa16a2de44494ef5]) successfully deleted
I20260929 06:29:43.707612 21459 ts_tablet_manager.cc:1918] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:29:43.707652 21459 log.cc:1196] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/117103bbaf464932948f09263808cacc
W20260929 06:29:43.707656 20910 catalog_manager.cc:4808] TS 4dbf593401a64172976745e95e780011 (127.13.15.131:44383): delete failed for tablet 117103bbaf464932948f09263808cacc because the tablet was not found. No further retry: Not found: Tablet not found: 117103bbaf464932948f09263808cacc
I20260929 06:29:43.707875 21459 ts_tablet_manager.cc:1939] T 117103bbaf464932948f09263808cacc P 2dc526a0efba4786a173766eaa1227d0: Deleting consensus metadata
I20260929 06:29:43.707890 20908 catalog_manager.cc:4841] TS cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781): tablet 117103bbaf464932948f09263808cacc (table test-table [id=5b57784586e046a7aa16a2de44494ef5]) successfully deleted
I20260929 06:29:43.708204 20910 catalog_manager.cc:4841] TS 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045): tablet 117103bbaf464932948f09263808cacc (table test-table [id=5b57784586e046a7aa16a2de44494ef5]) successfully deleted
W20260929 06:29:43.708280 20910 catalog_manager.cc:4808] TS 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045): delete failed for tablet 117103bbaf464932948f09263808cacc because the tablet was not found. No further retry: Not found: Tablet not found: 117103bbaf464932948f09263808cacc
I20260929 06:29:43.708575 21060 tablet_service.cc:1467] Processing CreateTablet for tablet fb6b4c8405574868bad6bd644883591b (DEFAULT_TABLE table=test-table [id=d8deb209b4e144dab7686c32a90830f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.708602 21200 tablet_service.cc:1467] Processing CreateTablet for tablet fb6b4c8405574868bad6bd644883591b (DEFAULT_TABLE table=test-table [id=d8deb209b4e144dab7686c32a90830f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.708680 21060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6b4c8405574868bad6bd644883591b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.708693 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6b4c8405574868bad6bd644883591b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.709836 21438 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: Bootstrap starting.
I20260929 06:29:43.709836 21434 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Bootstrap starting.
I20260929 06:29:43.710114 21438 tablet_bootstrap.cc:654] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.710404 21339 tablet_service.cc:1467] Processing CreateTablet for tablet fb6b4c8405574868bad6bd644883591b (DEFAULT_TABLE table=test-table [id=d8deb209b4e144dab7686c32a90830f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:29:43.710496 21339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6b4c8405574868bad6bd644883591b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:43.710518 21434 tablet_bootstrap.cc:654] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.711205 21434 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: No bootstrap required, opened a new log
I20260929 06:29:43.711251 21434 ts_tablet_manager.cc:1397] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:43.711378 21434 raft_consensus.cc:348] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.711434 21434 raft_consensus.cc:374] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.711454 21434 raft_consensus.cc:729] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb76ad97af27428ba2cd775a48487b72, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.711493 21434 consensus_queue.cc:260] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.711509 21435 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Bootstrap starting.
I20260929 06:29:43.711573 21434 ts_tablet_manager.cc:1428] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.711867 21435 tablet_bootstrap.cc:654] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:43.711992 21438 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: No bootstrap required, opened a new log
I20260929 06:29:43.712030 21438 ts_tablet_manager.cc:1397] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:29:43.712157 21438 raft_consensus.cc:348] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.712218 21438 raft_consensus.cc:374] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.712232 21438 raft_consensus.cc:729] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc526a0efba4786a173766eaa1227d0, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.712260 21438 consensus_queue.cc:260] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.712348 21438 ts_tablet_manager.cc:1428] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:43.712903 21435 tablet_bootstrap.cc:492] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: No bootstrap required, opened a new log
I20260929 06:29:43.712946 21435 ts_tablet_manager.cc:1397] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:29:43.713035 21435 raft_consensus.cc:348] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.713102 21435 raft_consensus.cc:374] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:43.713135 21435 raft_consensus.cc:729] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbf593401a64172976745e95e780011, State: Initialized, Role: FOLLOWER
I20260929 06:29:43.713176 21435 consensus_queue.cc:260] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:43.713269 21435 ts_tablet_manager.cc:1428] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:44.017138 21483 raft_consensus.cc:482] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:29:44.017282 21483 raft_consensus.cc:504] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:44.017493 21483 leader_election.cc:288] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:44.017707 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6b4c8405574868bad6bd644883591b" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0" is_pre_election: true
I20260929 06:29:44.017825 21077 raft_consensus.cc:2454] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:44.017832 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6b4c8405574868bad6bd644883591b" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72" is_pre_election: true
I20260929 06:29:44.017935 21220 raft_consensus.cc:2454] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 0.
I20260929 06:29:44.018023 21294 leader_election.cc:302] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc526a0efba4786a173766eaa1227d0, 4dbf593401a64172976745e95e780011; no voters:
I20260929 06:29:44.018157 21483 raft_consensus.cc:2781] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:29:44.018204 21483 raft_consensus.cc:482] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:29:44.018219 21483 raft_consensus.cc:3037] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:44.018930 21483 raft_consensus.cc:504] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:44.019071 21483 leader_election.cc:288] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Requested vote from peers 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045), cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781)
I20260929 06:29:44.019227 21220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6b4c8405574868bad6bd644883591b" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb76ad97af27428ba2cd775a48487b72"
I20260929 06:29:44.019214 21077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6b4c8405574868bad6bd644883591b" candidate_uuid: "4dbf593401a64172976745e95e780011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc526a0efba4786a173766eaa1227d0"
I20260929 06:29:44.019306 21220 raft_consensus.cc:3037] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:44.019305 21077 raft_consensus.cc:3037] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:44.019873 21220 raft_consensus.cc:2454] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:44.020054 21292 leader_election.cc:302] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbf593401a64172976745e95e780011, cb76ad97af27428ba2cd775a48487b72; no voters:
I20260929 06:29:44.020093 21077 raft_consensus.cc:2454] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbf593401a64172976745e95e780011 in term 1.
I20260929 06:29:44.020175 21483 raft_consensus.cc:2781] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:44.020242 21483 raft_consensus.cc:686] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbf593401a64172976745e95e780011, State: Running, Role: LEADER
I20260929 06:29:44.020320 21483 consensus_queue.cc:237] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } }
I20260929 06:29:44.020895 20923 catalog_manager.cc:5495] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbf593401a64172976745e95e780011 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "4dbf593401a64172976745e95e780011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dbf593401a64172976745e95e780011" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 44383 } health_report { overall_health: HEALTHY } } }
I20260929 06:29:44.067986 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:44.065836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.13.15.131:44383)
0929 06:29:44.065917 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:44.065919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:44.065927 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:44.065948 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:44.065949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:44.065966 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:44.066031 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:29:44.066185 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:44.066685 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:44.067452 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:44.067635 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:44.067687 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:44.067814 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:29:44.067840 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:29:44.067876 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:44.067908 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 06:29:44.067924 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"thread_start_us":45,"threads_started":1}
I20260929 06:29:44.070251 21077 raft_consensus.cc:1261] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:44.070282 21220 raft_consensus.cc:1261] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Refusing update from remote peer 4dbf593401a64172976745e95e780011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:29:44.070461 21441 consensus_queue.cc:1035] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc526a0efba4786a173766eaa1227d0" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 42045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:44.070552 21483 consensus_queue.cc:1035] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb76ad97af27428ba2cd775a48487b72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 32781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:29:44.080089 20923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:44.080219 20923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:29:44.080911 20923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0f265c1843490480a37c97b7ef30ae: Sending DeleteTablet for 3 replicas of tablet fb6b4c8405574868bad6bd644883591b
I20260929 06:29:44.081187 21060 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6b4c8405574868bad6bd644883591b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:44 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:38168
I20260929 06:29:44.081218 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6b4c8405574868bad6bd644883591b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:44 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57286
I20260929 06:29:44.081313 21459 tablet_replica.cc:330] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0: stopping tablet replica
I20260929 06:29:44.081355 21458 tablet_replica.cc:330] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: stopping tablet replica
I20260929 06:29:44.081372 21459 raft_consensus.cc:2229] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:44.081405 21459 raft_consensus.cc:2258] T fb6b4c8405574868bad6bd644883591b P 2dc526a0efba4786a173766eaa1227d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:44.081409 21458 raft_consensus.cc:2229] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:29:44.081445 21458 raft_consensus.cc:2258] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:44.081643 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20995
I20260929 06:29:44.083992 21458 ts_tablet_manager.cc:1905] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:44.085156 21339 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6b4c8405574868bad6bd644883591b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:29:44 UTC) from {username='oryx', principal='oryx/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:37506
I20260929 06:29:44.085273 21457 tablet_replica.cc:330] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: stopping tablet replica
I20260929 06:29:44.085335 21457 raft_consensus.cc:2229] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:29:44.085440 21457 raft_consensus.cc:2258] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:29:44.085574 21458 ts_tablet_manager.cc:1918] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:29:44.085632 21458 log.cc:1196] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/fb6b4c8405574868bad6bd644883591b
I20260929 06:29:44.085759 21457 ts_tablet_manager.cc:1905] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:29:44.085842 21458 ts_tablet_manager.cc:1939] T fb6b4c8405574868bad6bd644883591b P cb76ad97af27428ba2cd775a48487b72: Deleting consensus metadata
I20260929 06:29:44.086184 20908 catalog_manager.cc:4841] TS cb76ad97af27428ba2cd775a48487b72 (127.13.15.130:32781): tablet fb6b4c8405574868bad6bd644883591b (table test-table [id=d8deb209b4e144dab7686c32a90830f0]) successfully deleted
I20260929 06:29:44.087212 21457 ts_tablet_manager.cc:1918] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 06:29:44.087257 21457 log.cc:1196] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/fb6b4c8405574868bad6bd644883591b
I20260929 06:29:44.087488 21457 ts_tablet_manager.cc:1939] T fb6b4c8405574868bad6bd644883591b P 4dbf593401a64172976745e95e780011: Deleting consensus metadata
W20260929 06:29:44.087944 20910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:42045 (error 108)
I20260929 06:29:44.088099 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21135
I20260929 06:29:44.088191 20910 catalog_manager.cc:4841] TS 4dbf593401a64172976745e95e780011 (127.13.15.131:44383): tablet fb6b4c8405574868bad6bd644883591b (table test-table [id=d8deb209b4e144dab7686c32a90830f0]) successfully deleted
I20260929 06:29:44.089259 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:44.088329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:42045 (local address 127.0.0.1:38180)
0929 06:29:44.089191 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:44.089229 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:42045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":36,"threads_started":1}
W20260929 06:29:44.089367 20910 catalog_manager.cc:4584] TS 2dc526a0efba4786a173766eaa1227d0 (127.13.15.129:42045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb6b4c8405574868bad6bd644883591b: Network error: Client connection negotiation failed: client connection to 127.13.15.129:42045: connect: Connection refused (error 111)
I20260929 06:29:44.093526 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21275
I20260929 06:29:44.100129 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 20891
2026-09-29T06:29:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2720 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:44 dist-test-slave-sm0p krb5kdc[21520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:44 dist-test-slave-sm0p krb5kdc[21520](info): set up 2 sockets
Sep 29 06:29:44 dist-test-slave-sm0p krb5kdc[21520](info): commencing operation
krb5kdc: starting...
W20260929 06:29:45.126664 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:45 dist-test-slave-sm0p krb5kdc[21520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:45.245517 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45469
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:45.327121 21533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:45.327353 21533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:45.327389 21533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:45.328562 21533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:45.328626 21533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:45.328653 21533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:45.328684 21533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:45.328709 21533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:45.330219 21533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:45.330485 21533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:45.330708 21533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:45.335536 21538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:45.335592 21539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:45.335541 21541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:45.335749 21533 server_base.cc:1034] running on GCE node
I20260929 06:29:45.336066 21533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:45.336364 21533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:45.337528 21533 hybrid_clock.cc:648] HybridClock initialized: now 1790663385337511 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T06:29:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1242 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:29:45 dist-test-slave-sm0p krb5kdc[21546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:29:45 dist-test-slave-sm0p krb5kdc[21546](info): set up 2 sockets
Sep 29 06:29:45 dist-test-slave-sm0p krb5kdc[21546](info): commencing operation
krb5kdc: starting...
W20260929 06:29:46.378782 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:29:46.496793 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38747
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:38747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:46.578374 21559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:46.578588 21559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:46.578612 21559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:46.579694 21559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:46.579734 21559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:46.579751 21559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:46.579766 21559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:46.579778 21559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:46.581171 21559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:38747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:46.581372 21559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:46.581547 21559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:46.586464 21567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:46.586483 21564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:46.586537 21565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:46.586707 21559 server_base.cc:1034] running on GCE node
I20260929 06:29:46.586999 21559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:46.587286 21559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:46.588451 21559 hybrid_clock.cc:648] HybridClock initialized: now 1790663386588449 us; error 40 us; skew 500 ppm
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:46.590821 21559 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:46.591324 21559 webserver.cc:460] Webserver started at http://127.13.15.190:41019/ using document root <none> and password file <none>
I20260929 06:29:46.591548 21559 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:46.591616 21559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:46.591766 21559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:46.592779 21559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "af02806226c343fbb893b9de30f41983"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "bf82d2dbd34ddef760eac84be2d36eb3"
server_key_iv: "04be4e9899654140f9ed844df957f7df"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.593143 21559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "af02806226c343fbb893b9de30f41983"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "bf82d2dbd34ddef760eac84be2d36eb3"
server_key_iv: "04be4e9899654140f9ed844df957f7df"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.594485 21559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:46.595187 21574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.595350 21559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:46.595449 21559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "af02806226c343fbb893b9de30f41983"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "bf82d2dbd34ddef760eac84be2d36eb3"
server_key_iv: "04be4e9899654140f9ed844df957f7df"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.595520 21559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:46.608976 21559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:46.610980 21559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:46.611145 21559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:46.615209 21559 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:38747
I20260929 06:29:46.615239 21626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:38747 every 8 connection(s)
I20260929 06:29:46.615656 21559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:46.616511 21627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:29:46.618835 21627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Bootstrap starting.
I20260929 06:29:46.619407 21627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Neither blocks nor log segments found. Creating new log.
I20260929 06:29:46.619688 21627 log.cc:824] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:46.620353 21627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: No bootstrap required, opened a new log
I20260929 06:29:46.621744 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21559
I20260929 06:29:46.621857 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:29:46.622059 13374 external_mini_cluster.cc:1428] Setting key 95a8f8f1f967f4dd4ac0e261c8f94499
I20260929 06:29:46.622225 21627 raft_consensus.cc:348] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:46.622380 21627 raft_consensus.cc:374] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:29:46.622401 21627 raft_consensus.cc:729] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af02806226c343fbb893b9de30f41983, State: Initialized, Role: FOLLOWER
I20260929 06:29:46.622556 21627 consensus_queue.cc:260] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:46.622674 21627 raft_consensus.cc:388] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:46.622736 21627 raft_consensus.cc:482] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:46.622788 21627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:29:46.623661 21627 raft_consensus.cc:504] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:46.623780 21627 leader_election.cc:302] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af02806226c343fbb893b9de30f41983; no voters:
I20260929 06:29:46.623988 21627 leader_election.cc:288] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:29:46.624053 21632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:29:46.624219 21632 raft_consensus.cc:686] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 LEADER]: Becoming Leader. State: Replica: af02806226c343fbb893b9de30f41983, State: Running, Role: LEADER
I20260929 06:29:46.624231 21627 sys_catalog.cc:564] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:46.624364 21632 consensus_queue.cc:237] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:46.624719 21632 sys_catalog.cc:455] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af02806226c343fbb893b9de30f41983. Latest consensus state: current_term: 1 leader_uuid: "af02806226c343fbb893b9de30f41983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } } }
I20260929 06:29:46.624811 21632 sys_catalog.cc:458] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:46.624739 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af02806226c343fbb893b9de30f41983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } } }
I20260929 06:29:46.624960 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: This master's current role is: LEADER
W20260929 06:29:46.626050 21647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:29:46.626111 21647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:29:46.626174 21641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:46.626330 21641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:46.627769 21641 catalog_manager.cc:1347] Generated new cluster ID: 675aac4aa3504bcd8aeb798de1705e52
I20260929 06:29:46.627825 21641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:46.636322 21641 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:29:46.637010 21641 catalog_manager.cc:1504] Loading token signing keys...
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:46.640165 21641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Generated new TSK 0
I20260929 06:29:46.640370 21641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:29:46.641448 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:46.622615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.13.15.190:38747)
0929 06:29:46.622877 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:46.622886 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:46.622918 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:29:46.623035 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:46.623040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:46.623088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:46.623221 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:46.623899 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.625433 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.626615 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.627205 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.627352 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:46.638000 (+ 10648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:46.638004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:46.638005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:46.638017 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:46.639803 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.640280 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.640283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.640284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.640401 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.640609 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.640610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.640610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.640823 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:46.640938 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:46.641068 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 06:29:46.641178 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:29:46.666671 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:46.749440 21653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:46.749675 21653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:46.749716 21653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:46.750996 21653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:46.751060 21653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:46.751106 21653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:46.752578 21653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:46.752846 21653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:46.753074 21653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:46.757905 21659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:46.757934 21658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:46.758054 21653 server_base.cc:1034] running on GCE node
W20260929 06:29:46.757905 21661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:46.758379 21653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:46.758633 21653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:46.759783 21653 hybrid_clock.cc:648] HybridClock initialized: now 1790663386759776 us; error 31 us; skew 500 ppm
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:46.762416 21653 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:29:46.762935 21653 webserver.cc:460] Webserver started at http://127.13.15.129:44513/ using document root <none> and password file <none>
I20260929 06:29:46.763162 21653 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:46.763227 21653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:46.763367 21653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:46.764482 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "7951ddd1950b421090c8d5098e0bb6c6"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "1ded1f37cbfa9dab30ad2256190456d4"
server_key_iv: "af59a929b198ec18c566a143027fa43d"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.764848 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "7951ddd1950b421090c8d5098e0bb6c6"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "1ded1f37cbfa9dab30ad2256190456d4"
server_key_iv: "af59a929b198ec18c566a143027fa43d"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.766175 21653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:29:46.766942 21668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.767120 21653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:46.767213 21653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "7951ddd1950b421090c8d5098e0bb6c6"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "1ded1f37cbfa9dab30ad2256190456d4"
server_key_iv: "af59a929b198ec18c566a143027fa43d"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.767298 21653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:46.775535 21653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:46.777858 21653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:46.778024 21653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:46.778345 21653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:46.778650 21653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:46.778703 21653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.778736 21653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:46.778764 21653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.784379 21653 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:43789
I20260929 06:29:46.784446 21781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:43789 every 8 connection(s)
I20260929 06:29:46.784744 21653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:46.793001 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21653
I20260929 06:29:46.793115 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:29:46.793339 13374 external_mini_cluster.cc:1428] Setting key 37c7351de1d0b7811a87087c332e7cfe
I20260929 06:29:46.794353 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:46.785411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:34745 (local address 127.13.15.190:38747)
0929 06:29:46.785489 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:46.785491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:46.785713 (+ 222us) server_negotiation.cc:407] Connection header received
0929 06:29:46.786288 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:46.786289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:46.786301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:46.786341 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:46.787272 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.787783 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.788835 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.789053 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.789130 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:46.791379 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:46.791380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:46.791381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:46.791388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:46.792707 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.793255 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.793256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.793256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.793320 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.793622 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.793623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.793624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.793756 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:46.793837 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:46.794175 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 06:29:46.794263 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:29:46.794595 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:46.785417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:34745)
0929 06:29:46.785596 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:46.785631 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:46.786103 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:46.786433 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:46.786437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:46.786988 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:46.787168 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:46.787172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.787880 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.787881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:46.788705 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:46.788707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.789135 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.789136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:46.789261 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:46.789780 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:46.789789 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:46.791233 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:46.792802 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:46.792809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:46.792811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:46.793132 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:46.793402 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:46.793403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:46.793403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:46.793546 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:46.793846 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:46.793852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:46.794009 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 06:29:46.794136 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 06:29:46.794327 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260929 06:29:46.795113 21782 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38747
I20260929 06:29:46.795223 21782 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:46.795408 21782 heartbeater.cc:511] Master 127.13.15.190:38747 requested a full tablet report, sending...
I20260929 06:29:46.796061 21591 ts_manager.cc:194] Registered new tserver with Master: 7951ddd1950b421090c8d5098e0bb6c6 (127.13.15.129:43789)
I20260929 06:29:46.796705 21591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:34745
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:29:46.820554 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:46.901338 21787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:46.901585 21787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:46.901625 21787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:46.902844 21787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:46.902911 21787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:46.902956 21787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:46.904431 21787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:46.904696 21787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:46.904927 21787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:46.909972 21792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:46.909998 21793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:46.910000 21795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:46.910246 21787 server_base.cc:1034] running on GCE node
I20260929 06:29:46.910454 21787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:46.910701 21787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:46.911870 21787 hybrid_clock.cc:648] HybridClock initialized: now 1790663386911867 us; error 43 us; skew 500 ppm
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:46.914392 21787 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:29:46.914855 21787 webserver.cc:460] Webserver started at http://127.13.15.130:45401/ using document root <none> and password file <none>
I20260929 06:29:46.915091 21787 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:46.915158 21787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:46.915302 21787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:46.916543 21787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "7b79b4c113e34a8ab380f2216515124f"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "415929ee93a58a12dc810a181fc06a24"
server_key_iv: "2ad5a1de6a21a042e3267983f98070d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.916917 21787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "7b79b4c113e34a8ab380f2216515124f"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "415929ee93a58a12dc810a181fc06a24"
server_key_iv: "2ad5a1de6a21a042e3267983f98070d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.918262 21787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:29:46.919003 21802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.919168 21787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:46.919265 21787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "7b79b4c113e34a8ab380f2216515124f"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "415929ee93a58a12dc810a181fc06a24"
server_key_iv: "2ad5a1de6a21a042e3267983f98070d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:46.919349 21787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:46.941442 21787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:46.943569 21787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:46.943747 21787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:46.944021 21787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:46.944406 21787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:46.944463 21787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.944509 21787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:46.944532 21787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:46.950532 21787 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:45951
I20260929 06:29:46.950579 21915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:45951 every 8 connection(s)
I20260929 06:29:46.950927 21787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:46.955564 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21787
I20260929 06:29:46.955700 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:29:46.955972 13374 external_mini_cluster.cc:1428] Setting key 6b7303c4b98fa038f6ab203235ea400e
Sep 29 06:29:46 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663386, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:46.960937 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:46.951664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33983 (local address 127.13.15.190:38747)
0929 06:29:46.951736 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:46.951739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:46.951962 (+ 223us) server_negotiation.cc:407] Connection header received
0929 06:29:46.952430 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:46.952432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:46.952442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:46.952507 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:46.953434 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.953917 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.954941 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.955133 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.955217 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:46.957789 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:46.957791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:46.957791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:46.957799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:46.959400 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.959919 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.959920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.959920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.959976 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:46.960275 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:46.960275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:46.960275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:46.960381 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:46.960431 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:46.960764 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 06:29:46.960835 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:29:46.961236 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:46.951653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:33983)
0929 06:29:46.951818 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:46.951861 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:46.952343 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:46.952562 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:46.952565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:46.953143 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:46.953346 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:46.953351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.954004 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.954005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:46.954855 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:46.954857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:46.955228 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:46.955228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:46.955355 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:46.955982 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:46.955994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:46.957676 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:46.959505 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:46.959512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:46.959515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:46.959838 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:46.960047 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:46.960048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:46.960048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:46.960209 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:46.960443 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:46.960449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:46.960623 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 06:29:46.960748 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 06:29:46.960939 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260929 06:29:46.961781 21916 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38747
I20260929 06:29:46.961877 21916 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:46.962070 21916 heartbeater.cc:511] Master 127.13.15.190:38747 requested a full tablet report, sending...
I20260929 06:29:46.962502 21591 ts_manager.cc:194] Registered new tserver with Master: 7b79b4c113e34a8ab380f2216515124f (127.13.15.130:45951)
I20260929 06:29:46.962874 21591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:33983
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:29:46.980743 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:47.061445 21921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:47.061715 21921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:47.061753 21921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:47.063011 21921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:47.063097 21921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:47.063144 21921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:47.064548 21921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:47.064749 21921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:47.065001 21921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:47.070109 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.070179 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.070161 21921 server_base.cc:1034] running on GCE node
W20260929 06:29:47.070093 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.070550 21921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:47.070832 21921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:47.071991 21921 hybrid_clock.cc:648] HybridClock initialized: now 1790663387071985 us; error 32 us; skew 500 ppm
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:47.074581 21921 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:29:47.075121 21921 webserver.cc:460] Webserver started at http://127.13.15.131:39811/ using document root <none> and password file <none>
I20260929 06:29:47.075374 21921 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:47.075419 21921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:47.075585 21921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:29:47.076632 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "fc7ffa2da01d4d9e827153b3d54ddd96"
format_stamp: "Formatted at 2026-09-29 06:29:47 on dist-test-slave-sm0p"
server_key: "a96ab1743525fd59d2b549c2e6d9eb0a"
server_key_iv: "6b20591b7505458f5025a4fcc33d6473"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.076993 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "fc7ffa2da01d4d9e827153b3d54ddd96"
format_stamp: "Formatted at 2026-09-29 06:29:47 on dist-test-slave-sm0p"
server_key: "a96ab1743525fd59d2b549c2e6d9eb0a"
server_key_iv: "6b20591b7505458f5025a4fcc33d6473"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.078390 21921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:29:47.079111 21936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.079288 21921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:29:47.079378 21921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "fc7ffa2da01d4d9e827153b3d54ddd96"
format_stamp: "Formatted at 2026-09-29 06:29:47 on dist-test-slave-sm0p"
server_key: "a96ab1743525fd59d2b549c2e6d9eb0a"
server_key_iv: "6b20591b7505458f5025a4fcc33d6473"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.079452 21921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:47.116210 21921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:47.118643 21921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:47.118804 21921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:47.119050 21921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:47.119366 21921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:47.119400 21921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.119421 21921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:47.119474 21921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.125604 21921 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40135
I20260929 06:29:47.125681 22049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40135 every 8 connection(s)
I20260929 06:29:47.125984 21921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:29:47.134584 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.126752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:47599 (local address 127.13.15.190:38747)
0929 06:29:47.126797 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.126799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.127058 (+ 259us) server_negotiation.cc:407] Connection header received
0929 06:29:47.127518 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.127519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.127531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.127583 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.128484 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.128957 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.129938 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.130111 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.130207 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.132244 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:47.132245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:47.132245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:47.132251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:47.133437 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:47.133792 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:47.133792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:47.133793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:47.133839 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:47.134039 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:47.134039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:47.134039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:47.134179 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:47.134236 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:47.134463 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 06:29:47.134523 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:29:47.134791 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.126755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:47599)
0929 06:29:47.126936 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.126973 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.127438 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.127636 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.127639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.128209 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.128411 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.128417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.129080 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.129080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.129868 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.129869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.130222 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.130223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.130339 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.130871 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.130878 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.132166 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:29:47.133504 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:47.133509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:47.133511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:47.133729 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:47.133897 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:29:47.133897 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:29:47.133898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:29:47.133987 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:29:47.134237 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:29:47.134241 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:29:47.134356 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 06:29:47.134436 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:29:47.134548 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260929 06:29:47.135300 22050 heartbeater.cc:347] Connected to a master server at 127.13.15.190:38747
I20260929 06:29:47.135416 22050 heartbeater.cc:464] Registering TS with master...
I20260929 06:29:47.135591 22050 heartbeater.cc:511] Master 127.13.15.190:38747 requested a full tablet report, sending...
I20260929 06:29:47.135957 21591 ts_manager.cc:194] Registered new tserver with Master: fc7ffa2da01d4d9e827153b3d54ddd96 (127.13.15.131:40135)
I20260929 06:29:47.136070 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 21921
I20260929 06:29:47.136174 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:29:47.136358 13374 external_mini_cluster.cc:1428] Setting key 83409b5e1f0fd773f89f63e8ccf3c120
I20260929 06:29:47.136471 21591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:47599
I20260929 06:29:47.137168 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
Entry for principal oryx/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab.
WARNING: no policy specified for oryx/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
Entry for principal oryx/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab.
WARNING: no policy specified for oryx/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
Entry for principal oryx/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab.
I20260929 06:29:47.208932 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21653
I20260929 06:29:47.213459 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21787
I20260929 06:29:47.217902 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21921
I20260929 06:29:47.221915 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 21559
I20260929 06:29:47.226962 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38747
--webserver_interface=127.13.15.190
--webserver_port=41019
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:38747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:29:47.305886 22059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:47.306084 22059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:47.306107 22059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:47.307294 22059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:29:47.307336 22059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:47.307353 22059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:47.307365 22059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:29:47.307377 22059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:29:47.308739 22059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:38747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:38747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:47.308928 22059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:47.309100 22059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:47.313794 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.313815 22065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.313803 22064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.313978 22059 server_base.cc:1034] running on GCE node
I20260929 06:29:47.314278 22059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:47.314581 22059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:47.315759 22059 hybrid_clock.cc:648] HybridClock initialized: now 1790663387315737 us; error 46 us; skew 500 ppm
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:47.318384 22059 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:29:47.318852 22059 webserver.cc:460] Webserver started at http://127.13.15.190:41019/ using document root <none> and password file <none>
I20260929 06:29:47.319099 22059 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:47.319185 22059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:47.320500 22059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:47.321223 22074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.321406 22059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:47.321511 22059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "af02806226c343fbb893b9de30f41983"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "bf82d2dbd34ddef760eac84be2d36eb3"
server_key_iv: "04be4e9899654140f9ed844df957f7df"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.321846 22059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:47.336206 22059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:47.338168 22059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:47.338338 22059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:47.342510 22059 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:38747
I20260929 06:29:47.342557 22126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:38747 every 8 connection(s)
I20260929 06:29:47.342936 22059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:29:47.344352 22127 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:29:47.344983 22127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Bootstrap starting.
I20260929 06:29:47.351490 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22059
I20260929 06:29:47.351794 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:43789
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:38747
--webserver_port=44513
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:29:47.352891 22127 log.cc:824] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Log is configured to *not* fsync() on all Append() calls
I20260929 06:29:47.354657 22127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:29:47.354991 22127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Bootstrap complete.
I20260929 06:29:47.357144 22127 raft_consensus.cc:348] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:47.357378 22127 raft_consensus.cc:729] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: af02806226c343fbb893b9de30f41983, State: Initialized, Role: FOLLOWER
I20260929 06:29:47.357559 22127 consensus_queue.cc:260] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:47.357635 22127 raft_consensus.cc:388] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:29:47.357684 22127 raft_consensus.cc:482] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:29:47.357730 22127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:29:47.358740 22127 raft_consensus.cc:504] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:47.358860 22127 leader_election.cc:302] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af02806226c343fbb893b9de30f41983; no voters:
I20260929 06:29:47.359045 22127 leader_election.cc:288] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 06:29:47.359077 22131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 06:29:47.359186 22131 raft_consensus.cc:686] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [term 2 LEADER]: Becoming Leader. State: Replica: af02806226c343fbb893b9de30f41983, State: Running, Role: LEADER
I20260929 06:29:47.359297 22127 sys_catalog.cc:564] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:29:47.359445 22131 consensus_queue.cc:237] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } }
I20260929 06:29:47.359752 22132 sys_catalog.cc:455] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "af02806226c343fbb893b9de30f41983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } } }
I20260929 06:29:47.359759 22133 sys_catalog.cc:455] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af02806226c343fbb893b9de30f41983. Latest consensus state: current_term: 2 leader_uuid: "af02806226c343fbb893b9de30f41983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af02806226c343fbb893b9de30f41983" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 38747 } } }
I20260929 06:29:47.359944 22133 sys_catalog.cc:458] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:47.360162 22132 sys_catalog.cc:458] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983 [sys.catalog]: This master's current role is: LEADER
I20260929 06:29:47.360427 22140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:29:47.360931 22140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:29:47.361069 22140 catalog_manager.cc:1259] Loaded cluster ID: 675aac4aa3504bcd8aeb798de1705e52
I20260929 06:29:47.361109 22140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:29:47.362444 22140 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:29:47.362613 22140 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af02806226c343fbb893b9de30f41983: Loaded TSK: 0
I20260929 06:29:47.362814 22140 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:29:47.433310 22129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:47.433560 22129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:47.433585 22129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:47.434865 22129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:47.434907 22129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:47.434934 22129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:29:47.436338 22129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.129.keytab
--principal=oryx/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:43789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=44513
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:47.436569 22129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:47.436815 22129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:47.441819 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.441845 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.441846 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.442028 22129 server_base.cc:1034] running on GCE node
I20260929 06:29:47.442276 22129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:47.442520 22129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:47.443706 22129 hybrid_clock.cc:648] HybridClock initialized: now 1790663387443681 us; error 43 us; skew 500 ppm
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:47.446406 22129 init.cc:377] Logged in from keytab as oryx/127.13.15.129@KRBTEST.COM (short username oryx)
I20260929 06:29:47.446913 22129 webserver.cc:460] Webserver started at http://127.13.15.129:44513/ using document root <none> and password file <none>
I20260929 06:29:47.447166 22129 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:47.447250 22129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:47.448596 22129 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:47.449215 22163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.449452 22129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:29:47.449542 22129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "7951ddd1950b421090c8d5098e0bb6c6"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "1ded1f37cbfa9dab30ad2256190456d4"
server_key_iv: "af59a929b198ec18c566a143027fa43d"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.449976 22129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:47.461365 22129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:47.463752 22129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:47.463909 22129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:47.464169 22129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:47.464517 22129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:47.464564 22129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.464596 22129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:47.464620 22129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.471822 22129 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:43789
I20260929 06:29:47.471863 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:43789 every 8 connection(s)
I20260929 06:29:47.472265 22129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:29:47.476754 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22129
I20260929 06:29:47.477064 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:45951
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:38747
--webserver_port=45401
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.481125 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.473017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:42961)
0929 06:29:47.473200 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.473248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.473740 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.474193 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.474196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.474765 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.474950 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.474954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.476073 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.476074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.476946 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.476948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.477671 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.477672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.477797 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.478362 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.478370 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.480902 (+ 2532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
W20260929 06:29:47.481230 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.473081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:42961 (local address 127.13.15.190:38747)
0929 06:29:47.473290 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.473297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.473336 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:29:47.473949 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.473961 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.474002 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.474135 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.475073 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.475971 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.477038 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.477580 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.477699 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.481042 (+ 3343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:42961: BlockingRecv error: recv got EOF from 127.13.15.129:42961 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260929 06:29:47.481724 22277 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38747 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.15.190:38747: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.487557 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.481961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:54509)
0929 06:29:47.482009 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.482020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.482057 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.482219 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.482219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.482387 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.482445 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.482447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.483343 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.483345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.484246 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.484249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.484682 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.484683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.484766 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.485231 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.485237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.487464 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:29:47.487792 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.482003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:54509 (local address 127.13.15.190:38747)
0929 06:29:47.482039 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.482041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.482049 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:47.482115 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.482116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.482128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.482213 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.482523 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.483247 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.484348 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.484597 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.484669 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.487564 (+ 2895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:54509: BlockingRecv error: recv got EOF from 127.13.15.129:54509 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.492990 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.487814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:41171)
0929 06:29:47.487991 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.488002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.488045 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.488207 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.488208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.488372 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.488411 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.488412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.489289 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.489290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.490094 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.490096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.490592 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.490593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.490647 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.491100 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.491105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.492922 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
W20260929 06:29:47.493089 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.487975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:41171 (local address 127.13.15.190:38747)
0929 06:29:47.488098 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.488100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.488108 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:47.488129 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.488130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.488140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.488201 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.488499 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.489205 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.490227 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.490521 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.490586 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.493011 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:41171: BlockingRecv error: recv got EOF from 127.13.15.129:41171 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
W20260929 06:29:47.493113 22277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:29:47.564073 22281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:47.564335 22281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:47.564378 22281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:47.565649 22281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:47.565732 22281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:47.565779 22281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:29:47.567322 22281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.130.keytab
--principal=oryx/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:45951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=45401
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:47.567587 22281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:47.567874 22281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:47.573035 22289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.573122 22281 server_base.cc:1034] running on GCE node
W20260929 06:29:47.573052 22291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.573067 22288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.573441 22281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:47.573686 22281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:47.574834 22281 hybrid_clock.cc:648] HybridClock initialized: now 1790663387574818 us; error 30 us; skew 500 ppm
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:47.577152 22281 init.cc:377] Logged in from keytab as oryx/127.13.15.130@KRBTEST.COM (short username oryx)
I20260929 06:29:47.577594 22281 webserver.cc:460] Webserver started at http://127.13.15.130:45401/ using document root <none> and password file <none>
I20260929 06:29:47.577805 22281 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:47.577883 22281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:47.579226 22281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:47.579782 22298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.579937 22281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:29:47.579995 22281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "7b79b4c113e34a8ab380f2216515124f"
format_stamp: "Formatted at 2026-09-29 06:29:46 on dist-test-slave-sm0p"
server_key: "415929ee93a58a12dc810a181fc06a24"
server_key_iv: "2ad5a1de6a21a042e3267983f98070d3"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.580348 22281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:47.601032 22281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:47.603358 22281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:47.603524 22281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:47.603768 22281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:47.604071 22281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:47.604122 22281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.604161 22281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:47.604187 22281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.609826 22281 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:45951
I20260929 06:29:47.609952 22411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:45951 every 8 connection(s)
I20260929 06:29:47.610292 22281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:29:47.613682 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22281
I20260929 06:29:47.613981 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:40135
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:38747
--webserver_port=39811
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.618109 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.610999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:55645)
0929 06:29:47.611170 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.611204 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.611683 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.611895 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.611898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.612458 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.612651 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.612656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.613304 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.613305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.614576 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.614579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.614929 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.614930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.615081 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.615651 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.615659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.617974 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
W20260929 06:29:47.618198 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.611027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:55645 (local address 127.13.15.190:38747)
0929 06:29:47.611104 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.611107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.611289 (+ 182us) server_negotiation.cc:407] Connection header received
0929 06:29:47.611765 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.611766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.611779 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.611830 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.612734 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.613226 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.614668 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.614850 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.614900 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.618089 (+ 3189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:55645: BlockingRecv error: recv got EOF from 127.13.15.130:55645 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 06:29:47.618505 22412 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38747 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.15.190:38747: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.623090 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.618742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:43315)
0929 06:29:47.618779 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.618789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.618856 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.618980 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.618980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.619123 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.619161 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.619162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.619785 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.619786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.620284 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.620285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.620629 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.620630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.620696 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.621155 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.621159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.623012 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260929 06:29:47.623139 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.618801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:43315 (local address 127.13.15.190:38747)
0929 06:29:47.618860 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.618862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.618871 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:47.618911 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.618912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.618921 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.618975 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.619244 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.619700 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.620367 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.620550 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.620630 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.623077 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:43315: BlockingRecv error: recv got EOF from 127.13.15.130:43315 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.627754 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.623381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:59877)
0929 06:29:47.623419 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.623428 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.623466 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.623585 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.623586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.623682 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.623704 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.623705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.624588 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.624589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.625086 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.625087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.625522 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.625523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.625559 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.625988 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.625993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.627660 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 06:29:47.627815 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.623417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:59877 (local address 127.13.15.190:38747)
0929 06:29:47.623464 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.623466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.623478 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:29:47.623500 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.623501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.623511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.623578 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.623755 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.624515 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.625152 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.625435 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.625516 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.627757 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:59877: BlockingRecv error: recv got EOF from 127.13.15.130:59877 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 06:29:47.627946 22412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:29:47.697626 22415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:29:47.697893 22415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:29:47.697930 22415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:29:47.699270 22415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:29:47.699337 22415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:29:47.699378 22415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:29:47.700788 22415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/krb5kdc/oryx_127.13.15.131.keytab
--principal=oryx/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:40135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=39811
--tserver_master_addrs=127.13.15.190:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:29:47.701037 22415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:29:47.701260 22415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:29:47.706120 22420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.706123 22421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:29:47.706130 22423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:29:47.706400 22415 server_base.cc:1034] running on GCE node
I20260929 06:29:47.706655 22415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:29:47.706898 22415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:29:47.708060 22415 hybrid_clock.cc:648] HybridClock initialized: now 1790663387708043 us; error 28 us; skew 500 ppm
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663387, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:29:47.710729 22415 init.cc:377] Logged in from keytab as oryx/127.13.15.131@KRBTEST.COM (short username oryx)
I20260929 06:29:47.711135 22415 webserver.cc:460] Webserver started at http://127.13.15.131:39811/ using document root <none> and password file <none>
I20260929 06:29:47.711330 22415 fs_manager.cc:359] Metadata directory not provided
I20260929 06:29:47.711387 22415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:29:47.712692 22415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:29:47.713213 22430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.713370 22415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.713450 22415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "fc7ffa2da01d4d9e827153b3d54ddd96"
format_stamp: "Formatted at 2026-09-29 06:29:47 on dist-test-slave-sm0p"
server_key: "a96ab1743525fd59d2b549c2e6d9eb0a"
server_key_iv: "6b20591b7505458f5025a4fcc33d6473"
server_key_version: "encryptionkey@0"
I20260929 06:29:47.713748 22415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:29:47.751438 22415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:29:47.753382 22415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:29:47.753572 22415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:29:47.753794 22415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:29:47.754150 22415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:29:47.754211 22415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.754249 22415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:29:47.754297 22415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:29:47.760468 22415 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40135
I20260929 06:29:47.760563 22543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40135 every 8 connection(s)
I20260929 06:29:47.760905 22415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 06:29:47.768148 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.761704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:34745 (local address 127.13.15.190:38747)
0929 06:29:47.761779 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.761781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.761998 (+ 217us) server_negotiation.cc:407] Connection header received
0929 06:29:47.762538 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.762539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.762551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.762590 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.763548 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.764042 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.765024 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.765230 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.765315 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.768104 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:34745: BlockingRecv error: recv got EOF from 127.13.15.131:34745 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 06:29:47.768163 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.761697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:34745)
0929 06:29:47.761870 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.761905 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.762429 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.762670 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.762674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.763232 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.763428 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.763433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.764135 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.764136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.764931 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.764933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.765324 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.765325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.765448 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.765898 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.765908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.768022 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
W20260929 06:29:47.768451 22544 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:38747 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.15.190:38747: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
I20260929 06:29:47.769920 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22415
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.773551 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.768710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:52257)
0929 06:29:47.768761 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.768771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.768815 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.769031 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.769032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.769167 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.769208 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.769209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.769933 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.769933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.770654 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.770657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.771010 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.771011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.771084 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.771503 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.771507 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.773476 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 06:29:47.774154 22280 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.768870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:52257 (local address 127.13.15.190:38747)
0929 06:29:47.768927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.768928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.768935 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:47.768953 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.768953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.768961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.769028 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.769306 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.769857 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.770711 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.770943 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.770991 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.774067 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:52257: BlockingRecv error: recv got EOF from 127.13.15.131:52257 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:47 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:47.777843 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.773911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:38153)
0929 06:29:47.773951 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:47.773960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:47.773992 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:47.774274 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:47.774275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:47.774403 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:47.774426 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.774427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.775034 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.775036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.775503 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:47.775504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.775878 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.775878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:47.775928 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.776258 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:47.776262 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:47.777750 (+ 1488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 06:29:47.777976 22544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:29:47.778025 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:47.770349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59532 (local address 127.13.15.190:38747)
0929 06:29:47.770409 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.770411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.770580 (+ 169us) server_negotiation.cc:407] Connection header received
0929 06:29:47.770658 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.770659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.770670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.770739 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.771459 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.771930 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.773034 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.773388 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.773448 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.774851 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:29:47.774856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:29:47.774856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:29:47.774868 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:29:47.776590 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:47.776970 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:47.776974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:47.776975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:47.777084 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:29:47.777275 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:29:47.777276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:29:47.777277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:29:47.777535 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:29:47.777692 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 06:29:47.777802 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 06:29:47.777915 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 06:29:47.778081 22548 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:47.773986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:38153 (local address 127.13.15.190:38747)
0929 06:29:47.774117 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:47.774118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:47.774126 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:47.774164 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:47.774164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:47.774175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:47.774264 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:47.774510 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.774962 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.775593 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:47.775791 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:47.775870 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:47.777875 (+ 2005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:38153: BlockingRecv error: recv got EOF from 127.13.15.131:38153 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:48.504124 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:48.493515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:53647)
0929 06:29:48.493748 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:48.493763 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:48.493820 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:48.493965 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:48.493966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:48.494120 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:48.494184 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.494185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.494788 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.494789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.495308 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.495309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.495631 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.495632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.495669 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.502168 (+ 6499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:48.502174 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:48.504057 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":178,"threads_started":1}
W20260929 06:29:48.504279 22550 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:48.493606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53647 (local address 127.13.15.190:38747)
0929 06:29:48.493727 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:48.493728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:48.493821 (+ 93us) server_negotiation.cc:407] Connection header received
0929 06:29:48.493866 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:48.493866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:48.493877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:48.493945 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:48.494235 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.494723 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.495391 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.495569 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.495618 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.504183 (+ 8565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:53647: BlockingRecv error: recv got EOF from 127.13.15.129:53647 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:48.632666 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:48.628278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:37933)
0929 06:29:48.628413 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:48.628423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:48.628475 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:48.628597 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:48.628597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:48.628768 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:48.628817 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.628819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.629448 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.629449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.629998 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.630000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.630328 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.630330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.630389 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.630825 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:48.630830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:48.632599 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
W20260929 06:29:48.632786 22550 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:48.628349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:37933 (local address 127.13.15.190:38747)
0929 06:29:48.628418 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:48.628420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:48.628465 (+ 45us) server_negotiation.cc:407] Connection header received
0929 06:29:48.628517 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:48.628517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:48.628527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:48.628576 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:48.628896 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.629383 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.630070 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.630260 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.630310 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.632688 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:37933: BlockingRecv error: recv got EOF from 127.13.15.130:37933 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:48 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:48.783241 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:48.778377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:38367)
0929 06:29:48.778534 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:48.778548 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:48.778668 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:48.778932 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:48.778934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:48.779161 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:48.779259 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.779262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.779968 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.779970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.780509 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:48.780510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.780817 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.780817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:48.780858 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.781263 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:48.781267 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:48.783131 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
W20260929 06:29:48.783252 22550 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:48.778477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:38367 (local address 127.13.15.190:38747)
0929 06:29:48.778548 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:48.778551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:48.778628 (+ 77us) server_negotiation.cc:407] Connection header received
0929 06:29:48.778789 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:48.778790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:48.778825 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:48.778891 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:48.779367 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.779883 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.780581 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:48.780756 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:48.780806 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:48.783198 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:38367: BlockingRecv error: recv got EOF from 127.13.15.131:38367 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:49.509110 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:49.504623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:47509)
0929 06:29:49.504789 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:49.504800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:49.504863 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:49.505010 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:49.505010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:49.505159 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:49.505206 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.505207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.505824 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.505825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.506446 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.506459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.506788 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.506788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.506833 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.507260 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:49.507265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:49.509019 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
W20260929 06:29:49.509267 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:49.504713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:47509 (local address 127.13.15.190:38747)
0929 06:29:49.504836 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:49.504838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:49.504847 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:49.504924 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:49.504925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:49.504936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:49.504994 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:49.505266 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.505759 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.506531 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.506720 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.506774 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.509118 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:47509: BlockingRecv error: recv got EOF from 127.13.15.129:47509 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":49,"threads_started":1}
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:49.637501 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:49.633158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:58099)
0929 06:29:49.633286 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:49.633303 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:49.633361 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:49.633513 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:49.633514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:49.633671 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:49.633719 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.633721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.634425 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.634426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.634945 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.634946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.635246 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.635247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.635283 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.635712 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:49.635717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:49.637444 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260929 06:29:49.637614 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:49.633216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:58099 (local address 127.13.15.190:38747)
0929 06:29:49.633293 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:49.633295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:49.633347 (+ 52us) server_negotiation.cc:407] Connection header received
0929 06:29:49.633416 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:49.633417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:49.633443 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:49.633487 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:49.633783 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.634346 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.635013 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.635188 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.635236 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.637538 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:58099: BlockingRecv error: recv got EOF from 127.13.15.130:58099 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:49 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:49.788300 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:49.783753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:47689)
0929 06:29:49.783920 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:49.783933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:49.783996 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:49.784133 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:49.784134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:49.784306 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:49.784354 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.784355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.784975 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.784976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.785493 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:49.785494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.785839 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.785840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:49.785891 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.786490 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:49.786496 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:49.788240 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
W20260929 06:29:49.788424 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:49.783899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:47689 (local address 127.13.15.190:38747)
0929 06:29:49.783971 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:49.783972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:49.783980 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:49.784053 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:49.784053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:49.784065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:49.784111 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:49.784413 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.784910 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.785561 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:49.785732 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:49.785781 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:49.788327 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:47689: BlockingRecv error: recv got EOF from 127.13.15.131:47689 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:50.514854 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:50.509702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:52651)
0929 06:29:50.509825 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:50.509836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:50.509930 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:50.510097 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:50.510098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:50.510318 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:50.510365 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.510367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.511043 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.511044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.511564 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.511566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.511909 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.511910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.511961 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.512618 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:50.512622 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:50.514756 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
W20260929 06:29:50.514930 22558 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:50.509753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:52651 (local address 127.13.15.190:38747)
0929 06:29:50.509847 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:50.509850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:50.509940 (+ 90us) server_negotiation.cc:407] Connection header received
0929 06:29:50.509988 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:50.509989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:50.510002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:50.510073 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:50.510448 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.510949 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.511639 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.511822 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.511871 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.514846 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:52651: BlockingRecv error: recv got EOF from 127.13.15.129:52651 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":46,"threads_started":1}
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:50.643009 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:50.638085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:33953)
0929 06:29:50.638289 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:50.638302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:50.638378 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:50.638610 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:50.638612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:50.638838 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:50.638924 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.638926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.639554 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.639555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.640073 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.640074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.640373 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.640374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.640413 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.641092 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:50.641096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:50.642954 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":132,"threads_started":1}
W20260929 06:29:50.643103 22558 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:50.638190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33953 (local address 127.13.15.190:38747)
0929 06:29:50.638276 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:50.638278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:50.638441 (+ 163us) server_negotiation.cc:407] Connection header received
0929 06:29:50.638470 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:50.638471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:50.638484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:50.638577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:50.639002 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.639487 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.640138 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.640313 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.640361 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.643039 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:33953: BlockingRecv error: recv got EOF from 127.13.15.130:33953 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:50 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:50.793100 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:50.788723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:54305)
0929 06:29:50.788870 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:50.788882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:50.788970 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:50.789141 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:50.789142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:50.789371 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:50.789464 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.789466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.790096 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.790097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.790620 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:50.790622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.790974 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.790975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:50.791013 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.791429 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:50.791433 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:50.793045 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
W20260929 06:29:50.793248 22558 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:50.788826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:54305 (local address 127.13.15.190:38747)
0929 06:29:50.788901 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:50.788904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:50.788929 (+ 25us) server_negotiation.cc:407] Connection header received
0929 06:29:50.789046 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:50.789047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:50.789061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:50.789119 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:50.789546 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.790034 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.790712 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:50.790904 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:50.790970 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:50.793156 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:54305: BlockingRecv error: recv got EOF from 127.13.15.131:54305 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:51.520073 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:51.515351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:34347)
0929 06:29:51.515495 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:51.515532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:51.515602 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:51.515735 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:51.515736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:51.515882 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:51.515952 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.515953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.516742 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.516743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.517294 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.517296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.517641 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.517642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.517681 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.518125 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:51.518149 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:51.519988 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
W20260929 06:29:51.520171 22562 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:51.515424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:34347 (local address 127.13.15.190:38747)
0929 06:29:51.515552 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:51.515553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:51.515587 (+ 34us) server_negotiation.cc:407] Connection header received
0929 06:29:51.515651 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:51.515651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:51.515662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:51.515723 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:51.516040 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.516608 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.517374 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.517573 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.517630 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.520082 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:34347: BlockingRecv error: recv got EOF from 127.13.15.129:34347 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:51.649142 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:51.643496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:53583)
0929 06:29:51.643617 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:51.643629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:51.643698 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:51.643833 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:51.643833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:51.644004 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:51.644050 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.644052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.644717 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.644718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.645679 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.645681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.646230 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.646231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.646295 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.646812 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:51.646818 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:51.649036 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
W20260929 06:29:51.649250 22562 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:51.643553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:53583 (local address 127.13.15.190:38747)
0929 06:29:51.643629 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:51.643631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:51.643678 (+ 47us) server_negotiation.cc:407] Connection header received
0929 06:29:51.643758 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:51.643759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:51.643770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:51.643826 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:51.644132 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.644632 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.645766 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.646161 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.646250 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.649184 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:53583: BlockingRecv error: recv got EOF from 127.13.15.130:53583 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:51 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:51.798383 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:51.793617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:54723)
0929 06:29:51.793758 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:51.793767 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:51.793840 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:51.793982 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:51.793983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:51.794196 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:51.794252 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.794254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.794888 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.794889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.795381 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:51.795382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.795747 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.795748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:51.795788 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.796392 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:51.796396 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:51.798327 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
W20260929 06:29:51.798506 22562 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:51.793695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:54723 (local address 127.13.15.190:38747)
0929 06:29:51.793752 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:51.793754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:51.793840 (+ 86us) server_negotiation.cc:407] Connection header received
0929 06:29:51.793898 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:51.793898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:51.793911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:51.793961 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:51.794317 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.794803 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.795476 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:51.795672 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:51.795738 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:51.798409 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:54723: BlockingRecv error: recv got EOF from 127.13.15.131:54723 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:52.525774 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:52.520699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:57195)
0929 06:29:52.520963 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:52.520974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:52.521026 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:52.521253 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:52.521254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:52.521483 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:52.521541 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.521544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.522257 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.522258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.522798 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.522800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.523164 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.523166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.523222 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.523677 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:52.523681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:52.525716 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
W20260929 06:29:52.525929 22566 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:52.520783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:57195 (local address 127.13.15.190:38747)
0929 06:29:52.520929 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:52.520950 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:52.521076 (+ 126us) server_negotiation.cc:407] Connection header received
0929 06:29:52.521101 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:52.521102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:52.521117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:52.521240 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:52.521624 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.522175 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.522901 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.523094 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.523168 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.525824 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:57195: BlockingRecv error: recv got EOF from 127.13.15.129:57195 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:52.654520 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:52.649806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:42611)
0929 06:29:52.649954 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:52.649965 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:52.650017 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:52.650262 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:52.650262 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:52.650469 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:52.650515 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.650517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.651246 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.651246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.651767 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.651769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.652070 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.652071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.652108 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.652531 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:52.652536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:52.654421 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":53,"threads_started":1}
W20260929 06:29:52.654572 22566 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:52.649980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:42611 (local address 127.13.15.190:38747)
0929 06:29:52.650053 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:52.650055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:52.650063 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:29:52.650087 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:52.650087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:52.650098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:52.650179 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:52.650579 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.651186 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.651836 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.652011 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.652059 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.654505 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:42611: BlockingRecv error: recv got EOF from 127.13.15.130:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:52 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:52.803732 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:52.798950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:41015)
0929 06:29:52.799109 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:52.799120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:52.799185 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:52.799313 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:52.799314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:52.799504 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:52.799546 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.799553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.800251 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.800253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.800784 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:52.800786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.801162 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.801163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:52.801249 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.801765 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:52.801770 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:52.803640 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
W20260929 06:29:52.803773 22566 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:52.799037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:41015 (local address 127.13.15.190:38747)
0929 06:29:52.799090 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:52.799091 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:52.799167 (+ 76us) server_negotiation.cc:407] Connection header received
0929 06:29:52.799237 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:52.799237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:52.799251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:52.799306 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:52.799651 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.800157 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.800868 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:52.801078 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:52.801162 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:52.803712 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:41015: BlockingRecv error: recv got EOF from 127.13.15.131:41015 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:53.535228 22569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:53.526281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:54693)
0929 06:29:53.526401 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:53.526411 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:53.526472 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:53.526615 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:53.526615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:53.526803 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:53.526851 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.526853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.527468 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.527469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.528092 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.528095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.528399 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.528400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.528437 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.533259 (+ 4822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:53.533263 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:53.535130 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
W20260929 06:29:53.535282 22570 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:53.526298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:54693 (local address 127.13.15.190:38747)
0929 06:29:53.526415 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:53.526417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:53.526456 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:29:53.526512 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:53.526513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:53.526523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:53.526610 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:53.526915 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.527399 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.528160 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.528341 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.528389 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.535212 (+ 6823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:54693: BlockingRecv error: recv got EOF from 127.13.15.129:54693 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":49,"threads_started":1}
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:53.659631 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:53.655031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:36775)
0929 06:29:53.655202 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:53.655230 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:53.655306 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:53.655457 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:53.655458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:53.655614 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:53.655663 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.655664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.656290 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.656291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.656813 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.656815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.657147 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.657147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.657185 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.657623 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:53.657627 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:53.659529 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
W20260929 06:29:53.659680 22570 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:53.655087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:36775 (local address 127.13.15.190:38747)
0929 06:29:53.655154 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:53.655157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:53.655303 (+ 146us) server_negotiation.cc:407] Connection header received
0929 06:29:53.655364 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:53.655365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:53.655377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:53.655426 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:53.655725 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.656219 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.656888 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.657083 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.657132 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.659616 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:36775: BlockingRecv error: recv got EOF from 127.13.15.130:36775 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:53 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:53.809731 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:53.804284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:45949)
0929 06:29:53.804412 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:53.804425 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:53.804493 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:53.804659 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:53.804660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:53.804839 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:53.804885 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.804887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.805598 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.805600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.806119 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:53.806120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.806565 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.806566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:53.806623 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.807177 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:53.807182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:53.809644 (+ 2462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
W20260929 06:29:53.809844 22570 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:53.804363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:45949 (local address 127.13.15.190:38747)
0929 06:29:53.804430 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:53.804432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:53.804478 (+ 46us) server_negotiation.cc:407] Connection header received
0929 06:29:53.804563 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:53.804563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:53.804577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:53.804653 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:53.805003 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.805500 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.806248 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:53.806460 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:53.806567 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:53.809745 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:45949: BlockingRecv error: recv got EOF from 127.13.15.131:45949 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:54.540504 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:54.535796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:41121)
0929 06:29:54.535949 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:54.535976 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:54.536037 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:54.536196 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:54.536197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:54.536426 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:54.536488 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.536490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.537124 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.537124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.537625 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.537626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.537938 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.537938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.537994 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.538415 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:54.538420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:54.540432 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
W20260929 06:29:54.540571 22574 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:54.535893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:41121 (local address 127.13.15.190:38747)
0929 06:29:54.536027 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:54.536029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:54.536061 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:29:54.536089 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:54.536090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:54.536103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:54.536176 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:54.536564 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.537057 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.537697 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.537876 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.537924 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.540519 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:41121: BlockingRecv error: recv got EOF from 127.13.15.129:41121 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":44,"threads_started":1}
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:54.664819 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:54.660113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:60293)
0929 06:29:54.660293 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:54.660304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:54.660374 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:54.660543 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:54.660544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:54.660727 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:54.660771 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.660773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.661444 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.661445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.662010 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.662011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.662426 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.662427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.662516 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.662974 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:54.662978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:54.664716 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
W20260929 06:29:54.664860 22574 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:54.660184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60293 (local address 127.13.15.190:38747)
0929 06:29:54.660267 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:54.660270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:54.660374 (+ 104us) server_negotiation.cc:407] Connection header received
0929 06:29:54.660457 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:54.660458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:54.660472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:54.660527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:54.660855 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.661356 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.662104 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.662347 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.662428 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.664797 (+ 2369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:60293: BlockingRecv error: recv got EOF from 127.13.15.130:60293 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:54 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:54.815037 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:54.810476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:60759)
0929 06:29:54.810631 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:54.810642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:54.810706 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:54.810834 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:54.810835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:54.811009 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:54.811057 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.811059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.811668 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.811669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.812180 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:54.812182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.812490 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.812491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:54.812528 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.813047 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:54.813052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:54.814972 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
W20260929 06:29:54.815109 22574 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:54.810533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:60759 (local address 127.13.15.190:38747)
0929 06:29:54.810605 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:54.810607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:54.810690 (+ 83us) server_negotiation.cc:407] Connection header received
0929 06:29:54.810756 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:54.810757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:54.810768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:54.810808 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:54.811124 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.811599 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.812252 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:54.812423 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:54.812476 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:54.815061 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:60759: BlockingRecv error: recv got EOF from 127.13.15.131:60759 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:55.545653 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:55.541012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:56623)
0929 06:29:55.541174 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:55.541184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:55.541229 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:55.541404 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:55.541405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:55.541557 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:55.541603 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.541604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.542239 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.542240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.542742 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.542743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.543069 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.543070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.543109 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.543632 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:55.543637 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:55.545586 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260929 06:29:55.545794 22578 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:55.541154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56623 (local address 127.13.15.190:38747)
0929 06:29:55.541275 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:55.541277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:55.541289 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:29:55.541311 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:55.541311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:55.541320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:55.541382 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:55.541668 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.542180 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.542813 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.542999 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.543051 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.545707 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:56623: BlockingRecv error: recv got EOF from 127.13.15.129:56623 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:55.669793 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:55.665306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:60215)
0929 06:29:55.665431 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:55.665444 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:55.665527 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:55.665658 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:55.665658 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:55.665872 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:55.665926 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.665929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.666781 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.666782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.667306 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.667308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.667627 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.667628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.667666 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.668060 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:55.668064 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:55.669737 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":41,"threads_started":1}
W20260929 06:29:55.669934 22578 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:55.665395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60215 (local address 127.13.15.190:38747)
0929 06:29:55.665437 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:55.665439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:55.665524 (+ 85us) server_negotiation.cc:407] Connection header received
0929 06:29:55.665588 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:55.665589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:55.665598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:55.665639 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:55.665996 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.666705 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.667380 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.667564 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.667612 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.669851 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:60215: BlockingRecv error: recv got EOF from 127.13.15.130:60215 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:55 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:55.820611 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:55.815611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:49547)
0929 06:29:55.815741 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:55.815753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:55.815817 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:55.815973 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:55.815974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:55.816158 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:55.816206 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.816207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.816898 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.816899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.817524 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:55.817527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.817842 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.817843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:55.817883 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.818456 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:55.818461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:55.820517 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
W20260929 06:29:55.820653 22578 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:55.815658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:49547 (local address 127.13.15.190:38747)
0929 06:29:55.815724 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:55.815726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:55.815801 (+ 75us) server_negotiation.cc:407] Connection header received
0929 06:29:55.815874 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:55.815876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:55.815907 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:55.815969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:55.816274 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.816754 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.817600 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:55.817775 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:55.817825 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:55.820592 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:49547: BlockingRecv error: recv got EOF from 127.13.15.131:49547 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:56.551254 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:56.546325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:56347)
0929 06:29:56.546469 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:56.546484 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:56.546556 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:56.546729 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:56.546729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:56.546908 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:56.546960 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.546962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.547843 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.547844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.548418 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.548420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.548877 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.548878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.548929 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.549341 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:56.549345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:56.551189 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
W20260929 06:29:56.551388 22582 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:56.546395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56347 (local address 127.13.15.190:38747)
0929 06:29:56.546541 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:56.546543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:56.546552 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:29:56.546627 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:56.546628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:56.546639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:56.546708 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:56.547033 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.547772 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.548493 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.548701 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.548750 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.551276 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:56347: BlockingRecv error: recv got EOF from 127.13.15.129:56347 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:56.674705 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:56.670289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:36493)
0929 06:29:56.670446 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:56.670457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:56.670524 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:56.670652 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:56.670652 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:56.670807 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:56.670856 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.670858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.671471 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.671472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.671994 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.671995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.672317 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.672318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.672357 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.672793 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:56.672797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:56.674610 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
W20260929 06:29:56.674757 22582 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:56.670350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:36493 (local address 127.13.15.190:38747)
0929 06:29:56.670413 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:56.670415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:56.670529 (+ 114us) server_negotiation.cc:407] Connection header received
0929 06:29:56.670575 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:56.670576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:56.670588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:56.670628 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:56.670920 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.671403 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.672064 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.672253 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.672301 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.674694 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:36493: BlockingRecv error: recv got EOF from 127.13.15.130:36493 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:56 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:56.825799 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:56.821091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:49765)
0929 06:29:56.821277 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:56.821289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:56.821364 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:56.821527 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:56.821528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:56.821706 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:56.821752 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.821754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.822516 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.822517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.823047 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:56.823048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.823451 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.823452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:56.823520 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.823938 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:56.823942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:56.825733 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
W20260929 06:29:56.825955 22582 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:56.821185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:49765 (local address 127.13.15.190:38747)
0929 06:29:56.821261 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:56.821264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:56.821365 (+ 101us) server_negotiation.cc:407] Connection header received
0929 06:29:56.821419 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:56.821420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:56.821434 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:56.821511 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:56.821839 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.822415 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.823139 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:56.823332 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:56.823427 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:56.825885 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:49765: BlockingRecv error: recv got EOF from 127.13.15.131:49765 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:57.556202 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:57.551698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:42539)
0929 06:29:57.551854 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:57.551865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:57.551929 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:57.552064 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:57.552065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:57.552212 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:57.552259 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.552261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.552891 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.552892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.553427 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.553429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.553826 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.553827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.553903 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.554359 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:57.554363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:57.556146 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260929 06:29:57.556360 22586 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:57.551750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:42539 (local address 127.13.15.190:38747)
0929 06:29:57.551872 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:57.551874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:57.551913 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:29:57.551973 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:57.551973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:57.551984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:57.552045 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:57.552323 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.552815 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.553510 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.553712 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.553812 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.556252 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:42539: BlockingRecv error: recv got EOF from 127.13.15.129:42539 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:57.680418 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:57.675281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:59839)
0929 06:29:57.675499 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:57.675527 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:57.675608 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:57.675786 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:57.675787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:57.675967 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:57.676013 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.676015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.676957 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.676959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.677494 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.677496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.677938 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.677939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.677991 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.678623 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:57.678628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:57.680354 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":131,"threads_started":1}
W20260929 06:29:57.680578 22586 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:57.675404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:59839 (local address 127.13.15.190:38747)
0929 06:29:57.675480 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:57.675482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:57.675590 (+ 108us) server_negotiation.cc:407] Connection header received
0929 06:29:57.675682 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:57.675683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:57.675697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:57.675776 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:57.676302 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.676827 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.677607 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.677811 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.677940 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.680484 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:59839: BlockingRecv error: recv got EOF from 127.13.15.130:59839 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:57 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:57.831017 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:57.826308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:39779)
0929 06:29:57.826421 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:57.826432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:57.826488 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:57.826681 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:57.826682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:57.826852 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:57.826906 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.826909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.827621 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.827622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.828146 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:57.828148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.828513 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.828514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:57.828569 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.829012 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:57.829016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:57.830959 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
W20260929 06:29:57.831162 22586 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:57.826410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:39779 (local address 127.13.15.190:38747)
0929 06:29:57.826488 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:57.826490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:57.826500 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:29:57.826548 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:57.826549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:57.826561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:57.826670 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:57.827029 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.827533 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.828237 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:57.828432 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:57.828516 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:57.831080 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:39779: BlockingRecv error: recv got EOF from 127.13.15.131:39779 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:58.566371 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:58.556691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:45139)
0929 06:29:58.556824 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:58.556834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:58.556878 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:58.556999 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:58.556999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:58.557160 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:58.557204 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.557205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.557824 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.557852 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.558691 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.558693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.558999 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.559000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.559044 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.564466 (+ 5422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:58.564470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:58.566306 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
W20260929 06:29:58.566514 22590 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:58.556743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:45139 (local address 127.13.15.190:38747)
0929 06:29:58.556887 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:58.556889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:58.556896 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:29:58.556917 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:58.556918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:58.556927 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:58.556985 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:58.557265 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.557735 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.558753 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.558938 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.558991 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.566421 (+ 7430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:45139: BlockingRecv error: recv got EOF from 127.13.15.129:45139 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:58.685645 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:58.680883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:60249)
0929 06:29:58.681064 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:58.681078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:58.681170 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:58.681316 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:58.681317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:58.681496 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:58.681550 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.681552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.682267 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.682268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.682820 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.682822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.683249 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.683250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.683309 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.683728 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:58.683732 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:58.685543 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
W20260929 06:29:58.685675 22590 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:58.680988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60249 (local address 127.13.15.190:38747)
0929 06:29:58.681072 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:58.681074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:58.681152 (+ 78us) server_negotiation.cc:407] Connection header received
0929 06:29:58.681228 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:58.681229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:58.681241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:58.681291 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:58.681638 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.682187 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.682948 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.683160 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.683249 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.685614 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:60249: BlockingRecv error: recv got EOF from 127.13.15.130:60249 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:58 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:58.835794 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:58.831466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:50065)
0929 06:29:58.831605 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:58.831622 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:58.831679 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:58.831803 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:58.831803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:58.831968 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:58.832014 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.832015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.832629 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.832630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.833151 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:58.833152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.833450 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.833451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:58.833489 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.833993 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:58.833998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:58.835729 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
W20260929 06:29:58.835867 22590 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:58.831556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:50065 (local address 127.13.15.190:38747)
0929 06:29:58.831597 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:58.831599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:58.831683 (+ 84us) server_negotiation.cc:407] Connection header received
0929 06:29:58.831721 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:58.831721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:58.831733 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:58.831783 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:58.832074 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.832565 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.833219 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:58.833389 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:58.833438 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:58.835820 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:50065: BlockingRecv error: recv got EOF from 127.13.15.131:50065 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:59.571798 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:59.566856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:42177)
0929 06:29:59.567028 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:59.567039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:59.567084 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:59.567240 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:59.567240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:59.567404 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:59.567447 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.567448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.568115 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.568116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.568659 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.568660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.569151 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.569152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.569220 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.569783 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:59.569787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:59.571727 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
W20260929 06:29:59.571934 22594 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:59.566957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:42177 (local address 127.13.15.190:38747)
0929 06:29:59.567060 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:59.567062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:59.567088 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:29:59.567157 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:59.567157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:59.567168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:59.567223 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:59.567526 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.568044 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.568760 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.569070 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.569153 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.571822 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:42177: BlockingRecv error: recv got EOF from 127.13.15.129:42177 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":44,"threads_started":1}
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:59.691377 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:59.686157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:45863)
0929 06:29:59.686357 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:59.686368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:59.686465 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:59.686629 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:59.686630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:59.686835 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:59.686876 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.686878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.687595 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.687596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.688467 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.688469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.688826 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.688827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.688878 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.689375 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:59.689380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:59.691270 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":112,"threads_started":1}
W20260929 06:29:59.691413 22594 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:59.686265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:45863 (local address 127.13.15.190:38747)
0929 06:29:59.686352 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:59.686355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:59.686447 (+ 92us) server_negotiation.cc:407] Connection header received
0929 06:29:59.686556 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:59.686557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:59.686572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:59.686624 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:59.686970 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.687496 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.688556 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.688750 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.688828 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.691349 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:45863: BlockingRecv error: recv got EOF from 127.13.15.130:45863 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:29:59 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:29:59.841133 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:29:59.836288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:39551)
0929 06:29:59.836431 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 06:29:59.836443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:29:59.836503 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:29:59.836639 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:29:59.836639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:29:59.836798 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:29:59.836842 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.836843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.837447 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.837447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.837977 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:29:59.837978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.838350 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.838351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:29:59.838396 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.839029 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:29:59.839035 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:29:59.841027 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
W20260929 06:29:59.841148 22594 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:29:59.836369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:39551 (local address 127.13.15.190:38747)
0929 06:29:59.836415 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:29:59.836417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:29:59.836511 (+ 94us) server_negotiation.cc:407] Connection header received
0929 06:29:59.836556 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:29:59.836557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:29:59.836569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:29:59.836619 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:29:59.836908 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.837382 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.838067 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:29:59.838272 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:29:59.838339 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:29:59.841091 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:39551: BlockingRecv error: recv got EOF from 127.13.15.131:39551 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:00.576506 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:00.572241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:48131)
0929 06:30:00.572390 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:00.572401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:00.572452 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:00.572563 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:00.572564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:00.572718 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:00.572764 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.572766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.573391 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.573392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.573898 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.573900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.574293 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.574294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.574337 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.574735 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:00.574739 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:00.576447 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260929 06:30:00.576656 22598 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:00.572325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:48131 (local address 127.13.15.190:38747)
0929 06:30:00.572430 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:00.572432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:00.572441 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:30:00.572496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:00.572496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:00.572506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:00.572560 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:00.572824 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.573322 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.573967 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.574235 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.574283 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.576563 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:48131: BlockingRecv error: recv got EOF from 127.13.15.129:48131 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":55,"threads_started":1}
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:00.696528 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:00.691966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:51315)
0929 06:30:00.692104 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:00.692116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:00.692177 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:00.692355 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:00.692356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:00.692538 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:00.692586 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.692589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.693261 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.693262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.693790 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.693791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.694188 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.694188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.694256 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.694707 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:00.694711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:00.696469 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260929 06:30:00.696700 22598 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:00.692076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:51315 (local address 127.13.15.190:38747)
0929 06:30:00.692145 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:00.692147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:00.692160 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:30:00.692263 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:00.692264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:00.692276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:00.692350 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:00.692682 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.693178 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.693881 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.694086 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.694184 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.696580 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:51315: BlockingRecv error: recv got EOF from 127.13.15.130:51315 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:00 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:00.846269 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:00.841741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:39875)
0929 06:30:00.841985 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:00.841997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:00.842062 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:00.842244 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:00.842245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:00.842421 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:00.842469 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.842471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.843103 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.843104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.843617 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:00.843618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.843936 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.843937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:00.843975 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.844452 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:00.844456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:00.846198 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":199,"threads_started":1}
W20260929 06:30:00.846328 22598 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:00.841792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:39875 (local address 127.13.15.190:38747)
0929 06:30:00.841839 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:00.841841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:00.842104 (+ 263us) server_negotiation.cc:407] Connection header received
0929 06:30:00.842126 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:00.842126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:00.842157 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:00.842200 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:00.842539 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.843018 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.843693 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:00.843871 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:00.843921 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:00.846278 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:39875: BlockingRecv error: recv got EOF from 127.13.15.131:39875 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:01.588817 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:01.576954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:46179)
0929 06:30:01.577085 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:01.577102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:01.577151 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:01.577339 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:01.577340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:01.577523 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:01.577570 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.577572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.578291 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.578291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.578816 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.578819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.579174 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.579175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.579212 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.586480 (+ 7268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:01.586488 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:01.588716 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260929 06:30:01.588878 22602 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:01.577023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:46179 (local address 127.13.15.190:38747)
0929 06:30:01.577156 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:01.577158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:01.577168 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:30:01.577229 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:01.577230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:01.577242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:01.577322 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:01.577645 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.578175 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.578892 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.579084 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.579156 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.588805 (+ 9649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:46179: BlockingRecv error: recv got EOF from 127.13.15.129:46179 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:01.702793 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:01.697008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:55223)
0929 06:30:01.697122 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:01.697134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:01.697202 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:01.697373 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:01.697374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:01.697623 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:01.697701 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.697703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.698803 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.698804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.699334 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.699336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.699873 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.699873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.699915 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.700426 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:01.700436 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:01.702677 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
W20260929 06:30:01.702867 22602 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:01.697062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:55223 (local address 127.13.15.190:38747)
0929 06:30:01.697127 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:01.697129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:01.697190 (+ 61us) server_negotiation.cc:407] Connection header received
0929 06:30:01.697269 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:01.697271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:01.697286 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:01.697359 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:01.697789 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.698714 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.699444 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.699791 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.699873 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.702777 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:55223: BlockingRecv error: recv got EOF from 127.13.15.130:55223 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:01 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.131@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:01.851720 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:01.846862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.131:38373)
0929 06:30:01.847059 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:01.847073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:01.847143 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:01.847314 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:01.847315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:01.847494 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:01.847536 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.847538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.848223 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.848224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.848871 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:01.848874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.849267 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.849267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:01.849322 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.849837 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:01.849843 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:01.851661 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
W20260929 06:30:01.851907 22602 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:01.846993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:38373 (local address 127.13.15.190:38747)
0929 06:30:01.847063 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:01.847066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:01.847140 (+ 74us) server_negotiation.cc:407] Connection header received
0929 06:30:01.847209 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:01.847210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:01.847223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:01.847291 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:01.847631 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.848134 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.848961 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:01.849173 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:01.849267 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:01.851748 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.131:38373: BlockingRecv error: recv got EOF from 127.13.15.131:38373 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 06:30:02 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:02 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.129@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:02.593789 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:02.589353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.129:55335)
0929 06:30:02.589491 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:02.589506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:02.589551 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:02.589703 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:02.589704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:02.589853 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:02.589899 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:02.589900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.590563 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.590563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:02.591085 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:02.591087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.591406 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.591407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:02.591445 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:02.591974 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:02.591978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:02.593722 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
W20260929 06:30:02.593917 22606 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:02.589419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:55335 (local address 127.13.15.190:38747)
0929 06:30:02.589535 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:02.589537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:02.589550 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:30:02.589604 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:02.589605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:02.589617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:02.589675 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:02.589960 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.590495 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.591162 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.591341 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.591388 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:02.593810 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.129:55335: BlockingRecv error: recv got EOF from 127.13.15.129:55335 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
Sep 29 06:30:02 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 06:30:02 dist-test-slave-sm0p krb5kdc[21546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.15.130@KRBTEST.COM for oryx/127.13.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 06:30:02.708425 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:02.703529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:38747 (local address 127.13.15.130:37435)
0929 06:30:02.703710 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:02.703727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:02.703784 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:02.703917 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:02.703918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:02.704111 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:02.704160 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:02.704162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.704810 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.704811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:02.705360 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:02.705361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.705794 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.705795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:02.705858 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:02.706500 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:02.706504 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:02.708366 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.15.190:38747: Server oryx/127.13.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
W20260929 06:30:02.708560 22606 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:02.703635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:37435 (local address 127.13.15.190:38747)
0929 06:30:02.703685 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:02.703687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:02.703792 (+ 105us) server_negotiation.cc:407] Connection header received
0929 06:30:02.703825 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:02.703825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:02.703836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:02.703894 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:02.704227 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.704713 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.705442 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.705710 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.705794 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:02.708476 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.15.130:37435: BlockingRecv error: recv got EOF from 127.13.15.130:37435 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:30:02.771021 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22129
I20260929 06:30:02.775513 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22281
I20260929 06:30:02.780048 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22415
I20260929 06:30:02.784433 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22059
2026-09-29T06:30:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17443 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T06:30:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:30:02Z Disabled control of system clock
I20260929 06:30:02.814687 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45083
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 06:30:02.896497 22611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:02.896739 22611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:02.896760 22611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:02.897905 22611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:30:02.897941 22611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:02.897954 22611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:02.897967 22611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:30:02.897979 22611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:30:02.899415 22611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:02.899637 22611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:02.899886 22611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:02.905397 22616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:02.905560 22617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:02.905367 22619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:02.905632 22611 server_base.cc:1034] running on GCE node
I20260929 06:30:02.905900 22611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:02.906359 22611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:02.907572 22611 hybrid_clock.cc:648] HybridClock initialized: now 1790663402907554 us; error 27 us; skew 500 ppm
I20260929 06:30:02.908905 22611 webserver.cc:460] Webserver started at http://127.13.15.190:35249/ using document root <none> and password file <none>
I20260929 06:30:02.909169 22611 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:02.909215 22611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:02.909309 22611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:02.910452 22611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "208239daa4ee4792a60a52af295f4a2f"
format_stamp: "Formatted at 2026-09-29 06:30:02 on dist-test-slave-sm0p"
server_key: "3a1d5abd574a4fb68d3ad4308a7be764"
server_key_iv: "3f0506fec472e433d0100acfd1104a4c"
server_key_version: "encryptionkey@0"
I20260929 06:30:02.910828 22611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "208239daa4ee4792a60a52af295f4a2f"
format_stamp: "Formatted at 2026-09-29 06:30:02 on dist-test-slave-sm0p"
server_key: "3a1d5abd574a4fb68d3ad4308a7be764"
server_key_iv: "3f0506fec472e433d0100acfd1104a4c"
server_key_version: "encryptionkey@0"
I20260929 06:30:02.912110 22611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:30:02.912823 22625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:02.913038 22611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:30:02.913139 22611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "208239daa4ee4792a60a52af295f4a2f"
format_stamp: "Formatted at 2026-09-29 06:30:02 on dist-test-slave-sm0p"
server_key: "3a1d5abd574a4fb68d3ad4308a7be764"
server_key_iv: "3f0506fec472e433d0100acfd1104a4c"
server_key_version: "encryptionkey@0"
I20260929 06:30:02.913220 22611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:02.932097 22611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:02.932464 22611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:02.932643 22611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:02.936806 22611 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:45083
I20260929 06:30:02.936913 22677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:45083 every 8 connection(s)
I20260929 06:30:02.937266 22611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:30:02.937928 22678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:02.939508 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22611
I20260929 06:30:02.939596 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:30:02.939816 13374 external_mini_cluster.cc:1428] Setting key 103770977d60659ca710fe1aa051cd4e
I20260929 06:30:02.941079 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f: Bootstrap starting.
I20260929 06:30:02.941834 22678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:02.942196 22678 log.cc:824] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:02.942953 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f: No bootstrap required, opened a new log
I20260929 06:30:02.945132 22678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } }
I20260929 06:30:02.945314 22678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:02.945354 22678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208239daa4ee4792a60a52af295f4a2f, State: Initialized, Role: FOLLOWER
I20260929 06:30:02.945492 22678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } }
I20260929 06:30:02.945600 22678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:30:02.945636 22678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:30:02.945691 22678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:02.946087 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:02.940395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.13.15.190:45083)
0929 06:30:02.940633 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:02.940644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:02.940680 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:30:02.941204 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:02.941213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:02.941294 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:02.941462 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:02.941646 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.943003 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.943833 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:02.944487 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:02.944646 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:02.945337 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:02.945341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:02.945341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:02.945555 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:02.945600 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:02.945628 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:02.945696 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:02.945825 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 06:30:02.945915 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260929 06:30:02.946524 22678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } }
I20260929 06:30:02.946655 22678 leader_election.cc:302] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 208239daa4ee4792a60a52af295f4a2f; no voters:
I20260929 06:30:02.946909 22678 leader_election.cc:288] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:30:02.946938 22683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:02.947050 22683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [term 1 LEADER]: Becoming Leader. State: Replica: 208239daa4ee4792a60a52af295f4a2f, State: Running, Role: LEADER
I20260929 06:30:02.947144 22683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } }
I20260929 06:30:02.947201 22678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:30:02.947535 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "208239daa4ee4792a60a52af295f4a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } } }
I20260929 06:30:02.947551 22685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 208239daa4ee4792a60a52af295f4a2f. Latest consensus state: current_term: 1 leader_uuid: "208239daa4ee4792a60a52af295f4a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "208239daa4ee4792a60a52af295f4a2f" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45083 } } }
I20260929 06:30:02.947640 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:02.947664 22685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:02.948132 22690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:30:02.948642 22690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:30:02.950250 22690 catalog_manager.cc:1347] Generated new cluster ID: 4fa83a4e01ac4b78b990c57b2159bb92
I20260929 06:30:02.950304 22690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:30:02.956524 22690 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:30:02.957237 22690 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:30:02.971340 22690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 208239daa4ee4792a60a52af295f4a2f: Generated new TSK 0
I20260929 06:30:02.971560 22690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:30:02.973120 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--builtin_ntp_servers=127.13.15.148:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 06:30:03.054443 22702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.054665 22702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.054688 22702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.055892 22702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.055934 22702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.055961 22702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:30:03.057367 22702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.057638 22702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.057956 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.063989 22708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.063989 22710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.064004 22707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.064689 22702 server_base.cc:1034] running on GCE node
I20260929 06:30:03.064909 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.065160 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.066323 22702 hybrid_clock.cc:648] HybridClock initialized: now 1790663403066287 us; error 52 us; skew 500 ppm
I20260929 06:30:03.067714 22702 webserver.cc:460] Webserver started at http://127.13.15.129:39767/ using document root <none> and password file <none>
I20260929 06:30:03.067970 22702 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.068040 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.068223 22702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.069301 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "13e36cadbbb34fbfbdbb6fbb8dc2ec1e"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "bfa6db56123257383ea563fb96caa30d"
server_key_iv: "b4d8aebdf7282ddb4d2baa14afc0d321"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.069669 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "13e36cadbbb34fbfbdbb6fbb8dc2ec1e"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "bfa6db56123257383ea563fb96caa30d"
server_key_iv: "b4d8aebdf7282ddb4d2baa14afc0d321"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.071138 22702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:03.071952 22716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.072148 22702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:30:03.072252 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "13e36cadbbb34fbfbdbb6fbb8dc2ec1e"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "bfa6db56123257383ea563fb96caa30d"
server_key_iv: "b4d8aebdf7282ddb4d2baa14afc0d321"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.072335 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.091501 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.091871 22702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.092046 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.092290 22702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.092648 22702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.092706 22702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.092748 22702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.092777 22702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.098702 22702 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:37311
I20260929 06:30:03.098743 22829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:37311 every 8 connection(s)
I20260929 06:30:03.099083 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:30:03.104218 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.099803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:54811 (local address 127.13.15.190:45083)
0929 06:30:03.099871 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.099874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.100144 (+ 270us) server_negotiation.cc:407] Connection header received
0929 06:30:03.100644 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.100645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.100655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.100703 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.101360 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.101863 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.102887 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.103071 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.103130 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.103979 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.103981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.103981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.103985 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.103999 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.104002 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.104044 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.104134 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.104156 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:30:03.104517 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.099799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45083 (local address 127.13.15.129:54811)
0929 06:30:03.100010 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.100045 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.100546 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.100766 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.100769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.100777 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.101285 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.101289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.101965 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.101966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.102809 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.102812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.103199 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.103200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.103330 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.103843 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.103843 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.103873 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.103898 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.104059 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.104064 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.104066 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.104121 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.104203 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260929 06:30:03.105050 22830 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45083
I20260929 06:30:03.105177 22830 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.105363 22830 heartbeater.cc:511] Master 127.13.15.190:45083 requested a full tablet report, sending...
I20260929 06:30:03.105885 22642 ts_manager.cc:194] Registered new tserver with Master: 13e36cadbbb34fbfbdbb6fbb8dc2ec1e (127.13.15.129:37311)
I20260929 06:30:03.106665 22642 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.129:54811
I20260929 06:30:03.108227 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22702
I20260929 06:30:03.108309 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:30:03.108474 13374 external_mini_cluster.cc:1428] Setting key 958cf17c38187d12148f49d1bce08927
I20260929 06:30:03.109362 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--builtin_ntp_servers=127.13.15.148:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 06:30:03.190864 22833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.191074 22833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.191095 22833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.192176 22833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.192212 22833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.192236 22833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:30:03.193475 22833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.193742 22833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.194015 22833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.199100 22839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.199102 22841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.199096 22838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.199357 22833 server_base.cc:1034] running on GCE node
I20260929 06:30:03.199600 22833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.199851 22833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.201025 22833 hybrid_clock.cc:648] HybridClock initialized: now 1790663403201019 us; error 39 us; skew 500 ppm
I20260929 06:30:03.202485 22833 webserver.cc:460] Webserver started at http://127.13.15.130:44415/ using document root <none> and password file <none>
I20260929 06:30:03.202751 22833 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.202829 22833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.202983 22833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.204144 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "c18f1823c542421997135710b711315d"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2515bf6d975fa8f3370d3785f09cf721"
server_key_iv: "ca00f53d8127f84fac15497d518eca8e"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.204517 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "c18f1823c542421997135710b711315d"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2515bf6d975fa8f3370d3785f09cf721"
server_key_iv: "ca00f53d8127f84fac15497d518eca8e"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.205767 22833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:30:03.206552 22847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.206740 22833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.206842 22833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "c18f1823c542421997135710b711315d"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2515bf6d975fa8f3370d3785f09cf721"
server_key_iv: "ca00f53d8127f84fac15497d518eca8e"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.206924 22833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.234270 22833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.234618 22833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.234777 22833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.235015 22833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.235360 22833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.235420 22833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.235457 22833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.235483 22833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.241315 22833 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:35519
I20260929 06:30:03.241359 22960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:35519 every 8 connection(s)
I20260929 06:30:03.241714 22833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:30:03.244561 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22833
I20260929 06:30:03.244660 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:30:03.244853 13374 external_mini_cluster.cc:1428] Setting key 0f3f9547bd7582d91d271dafdab6dd0b
I20260929 06:30:03.245895 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--builtin_ntp_servers=127.13.15.148:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 06:30:03.247639 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.242515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:51129 (local address 127.13.15.190:45083)
0929 06:30:03.242559 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.242561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.242823 (+ 262us) server_negotiation.cc:407] Connection header received
0929 06:30:03.243299 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.243300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.243309 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.243365 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.243997 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.244482 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.245919 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.246243 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.246300 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.247265 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.247266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.247267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.247272 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.247291 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.247297 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.247358 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.247478 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.247506 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:30:03.247846 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.242476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45083 (local address 127.13.15.130:51129)
0929 06:30:03.242691 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.242732 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.243220 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.243426 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.243429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.243437 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.243928 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.243933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.244589 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.244590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.245838 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.245841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.246338 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.246340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.246507 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.247079 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.247080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.247120 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.247149 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.247377 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.247383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.247387 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.247465 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.247592 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260929 06:30:03.248474 22961 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45083
I20260929 06:30:03.258263 22961 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.258484 22961 heartbeater.cc:511] Master 127.13.15.190:45083 requested a full tablet report, sending...
I20260929 06:30:03.258966 22642 ts_manager.cc:194] Registered new tserver with Master: c18f1823c542421997135710b711315d (127.13.15.130:35519)
I20260929 06:30:03.259462 22642 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.130:51129
W20260929 06:30:03.327791 22964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.328048 22964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.328086 22964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.329242 22964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.329320 22964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.329361 22964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:30:03.330732 22964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:45083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.331010 22964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.331333 22964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.336566 22970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.336542 22969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.336576 22964 server_base.cc:1034] running on GCE node
W20260929 06:30:03.336536 22972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.336958 22964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.337162 22964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.338299 22964 hybrid_clock.cc:648] HybridClock initialized: now 1790663403338261 us; error 55 us; skew 500 ppm
I20260929 06:30:03.339485 22964 webserver.cc:460] Webserver started at http://127.13.15.131:39225/ using document root <none> and password file <none>
I20260929 06:30:03.339751 22964 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.339800 22964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.339929 22964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.340979 22964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "639ab298cce443fca3903ea0bfa9b310"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "d81ec737fc0398e8a53339ba04f34a8d"
server_key_iv: "ae4a0a522edadb4887d7b25b8616e945"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.341363 22964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "639ab298cce443fca3903ea0bfa9b310"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "d81ec737fc0398e8a53339ba04f34a8d"
server_key_iv: "ae4a0a522edadb4887d7b25b8616e945"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.342731 22964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:03.343420 22978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.343605 22964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:30:03.343701 22964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "639ab298cce443fca3903ea0bfa9b310"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "d81ec737fc0398e8a53339ba04f34a8d"
server_key_iv: "ae4a0a522edadb4887d7b25b8616e945"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.343772 22964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.355937 22964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.356245 22964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.356400 22964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.356637 22964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.356952 22964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.356984 22964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.357007 22964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.357059 22964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.362694 22964 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:44389
I20260929 06:30:03.362769 23091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:44389 every 8 connection(s)
I20260929 06:30:03.363106 22964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:30:03.368253 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.363950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:38341 (local address 127.13.15.190:45083)
0929 06:30:03.364018 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.364020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.364224 (+ 204us) server_negotiation.cc:407] Connection header received
0929 06:30:03.364707 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.364708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.364720 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.364790 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.365435 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.365929 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.366979 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.367187 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.367255 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.368000 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.368001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.368001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.368006 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.368024 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.368028 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.368079 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.368171 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.368193 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:30:03.368443 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.363858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45083 (local address 127.13.15.131:38341)
0929 06:30:03.364067 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.364108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.364581 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.364864 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.364868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.364876 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.365338 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.365342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.366012 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.366013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.366881 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:03.366883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:03.367269 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:03.367270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:03.367371 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:03.367860 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.367861 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.367887 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.367909 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.368093 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.368097 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.368099 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.368149 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.368222 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260929 06:30:03.369093 23092 heartbeater.cc:347] Connected to a master server at 127.13.15.190:45083
I20260929 06:30:03.369186 23092 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.369408 23092 heartbeater.cc:511] Master 127.13.15.190:45083 requested a full tablet report, sending...
I20260929 06:30:03.369853 22642 ts_manager.cc:194] Registered new tserver with Master: 639ab298cce443fca3903ea0bfa9b310 (127.13.15.131:44389)
I20260929 06:30:03.370285 22642 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.131:38341
I20260929 06:30:03.371800 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 22964
I20260929 06:30:03.371873 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:30:03.372022 13374 external_mini_cluster.cc:1428] Setting key f234ed1dd629b2c28f1913902ed960a7
I20260929 06:30:03.372666 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 06:30:03.374513 22681 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:03.373942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.13.15.190:45083)
0929 06:30:03.373999 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.374001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.374110 (+ 109us) server_negotiation.cc:407] Connection header received
0929 06:30:03.374195 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.374195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.374204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.374263 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.374408 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53818: BlockingRecv error: recv got EOF from 127.0.0.1:53818 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:30:03.374598 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22702
I20260929 06:30:03.379823 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22833
I20260929 06:30:03.383934 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22964
I20260929 06:30:03.387960 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 22611
2026-09-29T06:30:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (599 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T06:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:30:03Z Disabled control of system clock
I20260929 06:30:03.414108 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:42003
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:42003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 06:30:03.493836 23103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.494050 23103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.494071 23103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.495265 23103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:30:03.495309 23103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.495322 23103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.495369 23103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:30:03.495397 23103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:30:03.496733 23103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:42003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:42003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.496997 23103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.497306 23103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.502316 23109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.502353 23111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.502316 23108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.502652 23103 server_base.cc:1034] running on GCE node
I20260929 06:30:03.502903 23103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.503186 23103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.504374 23103 hybrid_clock.cc:648] HybridClock initialized: now 1790663403504349 us; error 35 us; skew 500 ppm
I20260929 06:30:03.505584 23103 webserver.cc:460] Webserver started at http://127.13.15.190:41679/ using document root <none> and password file <none>
I20260929 06:30:03.505834 23103 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.505882 23103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.506050 23103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.507164 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "9454f1bcf81a42bc99060790bb366df6"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "3d3db9bbacea40fd6b599fe55612b734"
server_key_iv: "60a016ecc7238699885e700e7df7baa1"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.507529 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "9454f1bcf81a42bc99060790bb366df6"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "3d3db9bbacea40fd6b599fe55612b734"
server_key_iv: "60a016ecc7238699885e700e7df7baa1"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.508941 23103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:03.509796 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.509994 23103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:30:03.510098 23103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "9454f1bcf81a42bc99060790bb366df6"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "3d3db9bbacea40fd6b599fe55612b734"
server_key_iv: "60a016ecc7238699885e700e7df7baa1"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.510231 23103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.513542 23103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.513826 23103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.513973 23103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.517800 23103 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:42003
I20260929 06:30:03.517830 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:42003 every 8 connection(s)
I20260929 06:30:03.518220 23103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:30:03.518433 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23103
I20260929 06:30:03.518535 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:30:03.518738 13374 external_mini_cluster.cc:1428] Setting key 1717939186c06ad74173b5cf7c389d1e
I20260929 06:30:03.518975 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:03.521577 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.519589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.13.15.190:42003)
0929 06:30:03.519784 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.519791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.519815 (+ 24us) server_negotiation.cc:407] Connection header received
0929 06:30:03.520275 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.520282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.520321 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.520432 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.521114 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.521118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.521118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.521122 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.521163 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.521173 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.521228 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.521332 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.521412 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260929 06:30:03.522078 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6: Bootstrap starting.
I20260929 06:30:03.522728 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:03.522948 23170 log.cc:824] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:03.523649 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6: No bootstrap required, opened a new log
I20260929 06:30:03.525249 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } }
I20260929 06:30:03.525406 23170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:03.525451 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9454f1bcf81a42bc99060790bb366df6, State: Initialized, Role: FOLLOWER
I20260929 06:30:03.525570 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } }
I20260929 06:30:03.525669 23170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:30:03.525708 23170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:30:03.525760 23170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:03.526355 23170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } }
I20260929 06:30:03.526466 23170 leader_election.cc:302] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9454f1bcf81a42bc99060790bb366df6; no voters:
I20260929 06:30:03.526631 23170 leader_election.cc:288] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:30:03.526885 23175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:03.526921 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:30:03.527027 23175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [term 1 LEADER]: Becoming Leader. State: Replica: 9454f1bcf81a42bc99060790bb366df6, State: Running, Role: LEADER
I20260929 06:30:03.527128 23175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } }
I20260929 06:30:03.527483 23177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9454f1bcf81a42bc99060790bb366df6. Latest consensus state: current_term: 1 leader_uuid: "9454f1bcf81a42bc99060790bb366df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } } }
I20260929 06:30:03.527617 23177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:03.527498 23176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9454f1bcf81a42bc99060790bb366df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9454f1bcf81a42bc99060790bb366df6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 42003 } } }
I20260929 06:30:03.527771 23176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6 [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:03.527824 23184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:30:03.528362 23184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:30:03.530073 23184 catalog_manager.cc:1347] Generated new cluster ID: 0b3b53a29df3449e8974d3383df4c222
I20260929 06:30:03.530148 23184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:30:03.537781 23184 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:30:03.537966 23184 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:30:03.550972 23184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9454f1bcf81a42bc99060790bb366df6: Generated new TSK 0
I20260929 06:30:03.551172 23184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:30:03.561020 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--builtin_ntp_servers=127.13.15.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 06:30:03.644384 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.644804 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.644857 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.646078 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.646181 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.646229 23194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:30:03.647868 23194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.648180 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.648519 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.654601 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.654620 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.654608 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.654847 23194 server_base.cc:1034] running on GCE node
I20260929 06:30:03.655056 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.655313 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.656498 23194 hybrid_clock.cc:648] HybridClock initialized: now 1790663403656486 us; error 29 us; skew 500 ppm
I20260929 06:30:03.663846 23194 webserver.cc:460] Webserver started at http://127.13.15.129:40987/ using document root <none> and password file <none>
I20260929 06:30:03.664129 23194 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.664206 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.664368 23194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.665496 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "513c7c676dee419980b0dbe7bef10957"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2b44f333856f1f763557443b8ea782ab"
server_key_iv: "e1691490bb9f6ec34ca603a884c952b9"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.665898 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "513c7c676dee419980b0dbe7bef10957"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2b44f333856f1f763557443b8ea782ab"
server_key_iv: "e1691490bb9f6ec34ca603a884c952b9"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.667338 23194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:30:03.668174 23208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.668399 23194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:30:03.668511 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "513c7c676dee419980b0dbe7bef10957"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "2b44f333856f1f763557443b8ea782ab"
server_key_iv: "e1691490bb9f6ec34ca603a884c952b9"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.668587 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.672173 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.672470 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.672619 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.672834 23194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.673163 23194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.673224 23194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.673265 23194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.673295 23194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.679200 23194 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:33579
I20260929 06:30:03.679280 23321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:33579 every 8 connection(s)
I20260929 06:30:03.679629 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:30:03.682693 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.680367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:33585 (local address 127.13.15.190:42003)
0929 06:30:03.680412 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.680415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.680661 (+ 246us) server_negotiation.cc:407] Connection header received
0929 06:30:03.681180 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.681181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.681189 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.681243 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.682434 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.682435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.682436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.682440 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.682454 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.682458 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.682506 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.682607 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.682621 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:30:03.682928 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.680339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:42003 (local address 127.13.15.129:33585)
0929 06:30:03.680515 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.680560 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.681061 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.681320 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.681324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.681332 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.682278 (+ 946us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.682279 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.682308 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.682336 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.682502 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.682506 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.682511 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.682619 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.682682 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260929 06:30:03.683426 23322 heartbeater.cc:347] Connected to a master server at 127.13.15.190:42003
I20260929 06:30:03.683502 23322 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.683821 23322 heartbeater.cc:511] Master 127.13.15.190:42003 requested a full tablet report, sending...
I20260929 06:30:03.684404 23134 ts_manager.cc:194] Registered new tserver with Master: 513c7c676dee419980b0dbe7bef10957 (127.13.15.129:33579)
I20260929 06:30:03.686261 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23194
I20260929 06:30:03.686360 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:30:03.686558 13374 external_mini_cluster.cc:1428] Setting key 016ed919af45355c1f7d6e11a48da881
I20260929 06:30:03.687469 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--builtin_ntp_servers=127.13.15.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 06:30:03.771763 23325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.771960 23325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.771982 23325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.773257 23325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.773301 23325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.773329 23325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:30:03.774718 23325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.774901 23325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.775146 23325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.780227 23330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.780233 23333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.780236 23331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.780251 23325 server_base.cc:1034] running on GCE node
I20260929 06:30:03.780639 23325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.780882 23325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.782040 23325 hybrid_clock.cc:648] HybridClock initialized: now 1790663403782026 us; error 44 us; skew 500 ppm
I20260929 06:30:03.783457 23325 webserver.cc:460] Webserver started at http://127.13.15.130:43067/ using document root <none> and password file <none>
I20260929 06:30:03.783706 23325 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.783752 23325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.783897 23325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.784946 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "189499193ceb45cf88a72c07211a0b42"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "ba1bfa8feaa4ef45eeb6eb459eabfd8c"
server_key_iv: "6eaa441268a614121e51db5741a27da3"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.785310 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "189499193ceb45cf88a72c07211a0b42"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "ba1bfa8feaa4ef45eeb6eb459eabfd8c"
server_key_iv: "6eaa441268a614121e51db5741a27da3"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.786643 23325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:03.787382 23339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.787554 23325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:30:03.787741 23325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "189499193ceb45cf88a72c07211a0b42"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "ba1bfa8feaa4ef45eeb6eb459eabfd8c"
server_key_iv: "6eaa441268a614121e51db5741a27da3"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.787920 23325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.791424 23325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.791690 23325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.791858 23325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.792073 23325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.792395 23325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.792452 23325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.792505 23325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.792536 23325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.798857 23325 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:37755
I20260929 06:30:03.798892 23452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:37755 every 8 connection(s)
I20260929 06:30:03.799228 23325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:30:03.802165 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23325
I20260929 06:30:03.802269 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:30:03.802392 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.800043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:54397 (local address 127.13.15.190:42003)
0929 06:30:03.800106 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.800110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.800327 (+ 217us) server_negotiation.cc:407] Connection header received
0929 06:30:03.800787 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.800788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.800793 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.800853 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.801986 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.801988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.801988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.801994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.802011 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.802016 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.802083 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.802238 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.802283 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:30:03.802491 13374 external_mini_cluster.cc:1428] Setting key 9031d0a5c08ec56fc49cc16fb481d7a6
I20260929 06:30:03.802562 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.800002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:42003 (local address 127.13.15.130:54397)
0929 06:30:03.800196 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.800237 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.800702 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.800905 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.800908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.800916 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.801797 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.801797 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.801825 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.801854 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.802121 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.802148 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.802152 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.802246 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.802312 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260929 06:30:03.803188 23453 heartbeater.cc:347] Connected to a master server at 127.13.15.190:42003
I20260929 06:30:03.803301 23453 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.803475 23453 heartbeater.cc:511] Master 127.13.15.190:42003 requested a full tablet report, sending...
I20260929 06:30:03.803848 23133 ts_manager.cc:194] Registered new tserver with Master: 189499193ceb45cf88a72c07211a0b42 (127.13.15.130:37755)
I20260929 06:30:03.803992 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--builtin_ntp_servers=127.13.15.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 06:30:03.887856 23456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:03.888067 23456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:03.888087 23456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:03.889123 23456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:03.889156 23456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:03.889180 23456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:30:03.890610 23456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:03.890826 23456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:03.891054 23456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:03.895886 23464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.895975 23456 server_base.cc:1034] running on GCE node
W20260929 06:30:03.895933 23461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:03.895885 23462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:03.896291 23456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:03.896528 23456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:03.897686 23456 hybrid_clock.cc:648] HybridClock initialized: now 1790663403897684 us; error 54 us; skew 500 ppm
I20260929 06:30:03.899040 23456 webserver.cc:460] Webserver started at http://127.13.15.131:35735/ using document root <none> and password file <none>
I20260929 06:30:03.899288 23456 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:03.899359 23456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:03.899513 23456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:03.900501 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "aa3b7d009add4bce99fa3c36061c1eed"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "26cc0d90c51491f46fd2d6b126b0ec49"
server_key_iv: "a1e70f64f0a9b720bbc1b9ca10b879f8"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.900799 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "aa3b7d009add4bce99fa3c36061c1eed"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "26cc0d90c51491f46fd2d6b126b0ec49"
server_key_iv: "a1e70f64f0a9b720bbc1b9ca10b879f8"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.901973 23456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:03.902710 23470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.902877 23456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.902961 23456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "aa3b7d009add4bce99fa3c36061c1eed"
format_stamp: "Formatted at 2026-09-29 06:30:03 on dist-test-slave-sm0p"
server_key: "26cc0d90c51491f46fd2d6b126b0ec49"
server_key_iv: "a1e70f64f0a9b720bbc1b9ca10b879f8"
server_key_version: "encryptionkey@0"
I20260929 06:30:03.903013 23456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:03.906219 23456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:03.906415 23456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:03.906502 23456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:03.906648 23456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:03.906881 23456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:03.906910 23456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.906929 23456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:03.906942 23456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:03.912644 23456 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:39195
I20260929 06:30:03.912719 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:39195 every 8 connection(s)
I20260929 06:30:03.913080 23456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:30:03.916054 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.913831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:51661 (local address 127.13.15.190:42003)
0929 06:30:03.913875 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.913878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.914119 (+ 241us) server_negotiation.cc:407] Connection header received
0929 06:30:03.914672 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.914673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.914679 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.914745 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.915868 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:03.915869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:03.915869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:30:03.915873 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:03.915888 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:30:03.915892 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:03.915943 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:03.915992 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 06:30:03.916003 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:30:03.916301 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:03.913819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:42003 (local address 127.13.15.131:51661)
0929 06:30:03.913999 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:03.914048 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:03.914557 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:03.914816 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:03.914820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:03.914828 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:03.915711 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:30:03.915712 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:03.915740 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:30:03.915768 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:03.915941 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:03.915943 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:03.915946 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:30:03.916017 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 06:30:03.916071 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260929 06:30:03.916836 23584 heartbeater.cc:347] Connected to a master server at 127.13.15.190:42003
I20260929 06:30:03.916941 23584 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:03.917083 23584 heartbeater.cc:511] Master 127.13.15.190:42003 requested a full tablet report, sending...
I20260929 06:30:03.917490 23133 ts_manager.cc:194] Registered new tserver with Master: aa3b7d009add4bce99fa3c36061c1eed (127.13.15.131:39195)
I20260929 06:30:03.919814 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23456
I20260929 06:30:03.919922 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:30:03.920135 13374 external_mini_cluster.cc:1428] Setting key 0ce627baef3ebbde45f8fc9b0c9ac663
I20260929 06:30:03.921006 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 06:30:03.922785 23173 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:30:03.922300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.13.15.190:42003)
0929 06:30:03.922337 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:03.922339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:03.922456 (+ 117us) server_negotiation.cc:407] Connection header received
0929 06:30:03.922535 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:03.922535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:03.922540 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:03.922590 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:03.922719 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42438: BlockingRecv error: recv got EOF from 127.0.0.1:42438 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:30:03.922848 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23194
I20260929 06:30:03.927542 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23325
I20260929 06:30:03.931770 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23456
I20260929 06:30:03.935714 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23103
2026-09-29T06:30:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (547 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:30:03 dist-test-slave-sm0p krb5kdc[23594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:30:03 dist-test-slave-sm0p krb5kdc[23594](info): set up 2 sockets
Sep 29 06:30:03 dist-test-slave-sm0p krb5kdc[23594](info): commencing operation
krb5kdc: starting...
W20260929 06:30:04.974650 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:30:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:30:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:30:05.101181 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43289
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:43289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:05.180765 23607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:05.180992 23607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:05.181016 23607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:05.182171 23607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:30:05.182209 23607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:05.182224 23607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:05.182236 23607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:30:05.182250 23607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:30:05.183674 23607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:43289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:05.183871 23607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:05.184043 23607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:05.189085 23615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.189090 23613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:05.189256 23607 server_base.cc:1034] running on GCE node
W20260929 06:30:05.189090 23612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:05.189731 23607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:05.190030 23607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:05.191177 23607 hybrid_clock.cc:648] HybridClock initialized: now 1790663405191163 us; error 29 us; skew 500 ppm
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:05.193765 23607 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:30:05.194329 23607 webserver.cc:460] Webserver started at http://127.13.15.190:41459/ using document root <none> and password file <none>
I20260929 06:30:05.194561 23607 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:05.194631 23607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:05.194792 23607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:05.195868 23607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "87d09665015a11341bed6dc8b6da74b8"
server_key_iv: "888f4a000e4277238b56638a84893d61"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.196252 23607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "87d09665015a11341bed6dc8b6da74b8"
server_key_iv: "888f4a000e4277238b56638a84893d61"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.197585 23607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:30:05.198410 23622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.198629 23607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:30:05.198750 23607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "87d09665015a11341bed6dc8b6da74b8"
server_key_iv: "888f4a000e4277238b56638a84893d61"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.198827 23607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:05.222600 23607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:05.225229 23607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:05.225414 23607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:05.229743 23607 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:43289
I20260929 06:30:05.229821 23674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:43289 every 8 connection(s)
I20260929 06:30:05.230225 23607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:30:05.231302 23675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:05.233515 23675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: Bootstrap starting.
I20260929 06:30:05.234068 23675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:05.234396 23675 log.cc:824] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:05.235129 23675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: No bootstrap required, opened a new log
I20260929 06:30:05.236172 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23607
I20260929 06:30:05.236279 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:30:05.236481 13374 external_mini_cluster.cc:1428] Setting key adfabc4f2b703b1e31c747e29cf05e92
I20260929 06:30:05.237253 23675 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } }
I20260929 06:30:05.237435 23675 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:05.237470 23675 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd89c7c72cef46f4a12ceb5ddb0fc6ed, State: Initialized, Role: FOLLOWER
I20260929 06:30:05.237592 23675 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } }
I20260929 06:30:05.237674 23675 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:30:05.237707 23675 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:30:05.237753 23675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:05.238927 23675 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } }
I20260929 06:30:05.239019 23675 leader_election.cc:302] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd89c7c72cef46f4a12ceb5ddb0fc6ed; no voters:
I20260929 06:30:05.239229 23675 leader_election.cc:288] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:30:05.239251 23680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:05.239367 23680 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [term 1 LEADER]: Becoming Leader. State: Replica: dd89c7c72cef46f4a12ceb5ddb0fc6ed, State: Running, Role: LEADER
I20260929 06:30:05.239454 23680 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } }
I20260929 06:30:05.239503 23675 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:30:05.239775 23681 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } } }
I20260929 06:30:05.239886 23681 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:05.239758 23682 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd89c7c72cef46f4a12ceb5ddb0fc6ed. Latest consensus state: current_term: 1 leader_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89c7c72cef46f4a12ceb5ddb0fc6ed" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43289 } } }
I20260929 06:30:05.240211 23682 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:05.240453 23685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:30:05.241012 23685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:30:05.242512 23685 catalog_manager.cc:1347] Generated new cluster ID: 3d395ecdf678429db3ccfbd41368babb
I20260929 06:30:05.242568 23685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:05.248598 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.237052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.13.15.190:43289)
0929 06:30:05.237245 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.237255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.237291 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:30:05.237413 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.237421 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.237481 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.237635 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:05.238626 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.239844 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.240932 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.241521 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.241631 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.244773 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:05.244777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:05.244777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:05.244791 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:05.246878 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.247274 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.247277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.247278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.247417 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.247621 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.247621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.247622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.247957 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:05.248148 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.248230 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.248395 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260929 06:30:05.254899 23685 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:30:05.255463 23685 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:30:05.258144 23685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: Generated new TSK 0
I20260929 06:30:05.258323 23685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:30:05.289865 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:05.371922 23701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:05.372138 23701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:05.372164 23701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:05.373339 23701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:05.373383 23701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:05.373410 23701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:30:05.374943 23701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:05.375231 23701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:05.375486 23701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:05.380697 23709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.380726 23707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.380697 23706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:05.380762 23701 server_base.cc:1034] running on GCE node
I20260929 06:30:05.381124 23701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:05.381384 23701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:05.382526 23701 hybrid_clock.cc:648] HybridClock initialized: now 1790663405382511 us; error 28 us; skew 500 ppm
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:05.385006 23701 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:30:05.385474 23701 webserver.cc:460] Webserver started at http://127.13.15.129:35057/ using document root <none> and password file <none>
I20260929 06:30:05.385710 23701 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:05.385756 23701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:05.385931 23701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:05.387075 23701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "eea3abbb641d452b88d263a581a3974b"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "12a62a9f17fc54332a73bc1da50ac5fc"
server_key_iv: "ecfe04a59b27f503ec7ccb82711dcbee"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.387445 23701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "eea3abbb641d452b88d263a581a3974b"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "12a62a9f17fc54332a73bc1da50ac5fc"
server_key_iv: "ecfe04a59b27f503ec7ccb82711dcbee"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.388754 23701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:30:05.389479 23716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.389658 23701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:30:05.389773 23701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "eea3abbb641d452b88d263a581a3974b"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "12a62a9f17fc54332a73bc1da50ac5fc"
server_key_iv: "ecfe04a59b27f503ec7ccb82711dcbee"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.389858 23701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:05.399344 23701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:05.401278 23701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:05.401444 23701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:05.401693 23701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:05.402011 23701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:05.402067 23701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.402110 23701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:05.402153 23701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.408039 23701 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:45773
I20260929 06:30:05.408113 23829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:45773 every 8 connection(s)
I20260929 06:30:05.408449 23701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:05.415221 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23701
I20260929 06:30:05.415365 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:30:05.415643 13374 external_mini_cluster.cc:1428] Setting key 388c00b53dd67e19005996378f20efd6
I20260929 06:30:05.418502 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.409243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:57623 (local address 127.13.15.190:43289)
0929 06:30:05.409350 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.409352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.409450 (+ 98us) server_negotiation.cc:407] Connection header received
0929 06:30:05.409922 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.409923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.409936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.409990 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:05.410999 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.411487 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.412491 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.412687 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.412737 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.415025 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:05.415027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:05.415028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:05.415039 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:05.416886 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.417333 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.417335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.417335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.417398 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.417718 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.417719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.417719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.417872 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:05.417945 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.418368 (+ 423us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.418438 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 06:30:05.418885 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.409181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43289 (local address 127.13.15.129:57623)
0929 06:30:05.409342 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.409378 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.409836 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.410042 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.410045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.410720 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:05.410917 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.410922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.411569 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.411569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.412412 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.412415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.412762 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.412763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.412858 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.413435 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:05.413443 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:05.414906 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:05.416986 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.416992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.416994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.417252 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.417507 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.417508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.417509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.417641 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.417955 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:05.417962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:05.418215 (+ 253us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.418347 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.418541 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260929 06:30:05.419471 23830 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43289
I20260929 06:30:05.419576 23830 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:05.419776 23830 heartbeater.cc:511] Master 127.13.15.190:43289 requested a full tablet report, sending...
I20260929 06:30:05.420281 23639 ts_manager.cc:194] Registered new tserver with Master: eea3abbb641d452b88d263a581a3974b (127.13.15.129:45773)
I20260929 06:30:05.420987 23639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:57623
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:30:05.440958 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:05.527266 23835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:05.527539 23835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:05.527580 23835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:05.528978 23835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:05.529057 23835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:05.529109 23835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:30:05.530797 23835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:05.531108 23835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:05.531387 23835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:05.537026 23840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.537050 23843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.537024 23841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:05.537262 23835 server_base.cc:1034] running on GCE node
I20260929 06:30:05.537570 23835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:05.537833 23835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:05.538997 23835 hybrid_clock.cc:648] HybridClock initialized: now 1790663405538981 us; error 54 us; skew 500 ppm
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:05.541750 23835 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:30:05.542246 23835 webserver.cc:460] Webserver started at http://127.13.15.130:42605/ using document root <none> and password file <none>
I20260929 06:30:05.542492 23835 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:05.542572 23835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:05.542721 23835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:05.543773 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "152221b172304bc5bc34ffd7c62aeba9"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "5d78ca0ca941f88c89b96a3fc687b4a5"
server_key_iv: "7ead634ba3a306d0387415e7136923c2"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.544142 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "152221b172304bc5bc34ffd7c62aeba9"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "5d78ca0ca941f88c89b96a3fc687b4a5"
server_key_iv: "7ead634ba3a306d0387415e7136923c2"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.545490 23835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:30:05.546322 23850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.546500 23835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:30:05.546600 23835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "152221b172304bc5bc34ffd7c62aeba9"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "5d78ca0ca941f88c89b96a3fc687b4a5"
server_key_iv: "7ead634ba3a306d0387415e7136923c2"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.546697 23835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:05.571426 23835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:05.573905 23835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:05.574097 23835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:05.574363 23835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:05.574700 23835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:05.574759 23835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.574792 23835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:05.574820 23835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.580513 23835 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:46685
I20260929 06:30:05.580555 23963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:46685 every 8 connection(s)
I20260929 06:30:05.580883 23835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:30:05.586045 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23835
I20260929 06:30:05.586195 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:30:05.586421 13374 external_mini_cluster.cc:1428] Setting key 7752e026836bd2a6a3934015ecad9e8f
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:05.590818 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.581574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60269 (local address 127.13.15.190:43289)
0929 06:30:05.581645 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.581647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.581873 (+ 226us) server_negotiation.cc:407] Connection header received
0929 06:30:05.582481 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.582483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.582495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.582551 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:05.583503 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.584013 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.585055 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.585273 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.585364 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.587885 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:05.587887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:05.587887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:05.587895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:05.589298 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.589786 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.589787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.589787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.589836 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.590112 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.590113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.590113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.590274 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:05.590326 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.590689 (+ 363us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.590756 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:30:05.591110 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.581575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43289 (local address 127.13.15.130:60269)
0929 06:30:05.581749 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.581785 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.582372 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.582626 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.582630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.583209 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:05.583402 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.583407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.584117 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.584118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.584950 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.584952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.585374 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.585375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.585521 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.586116 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:05.586124 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:05.587790 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:05.589385 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.589390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.589393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.589712 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.589906 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.589907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.589907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.590049 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.590350 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:05.590356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:05.590530 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.590661 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.590846 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260929 06:30:05.591619 23964 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43289
I20260929 06:30:05.591732 23964 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:05.591926 23964 heartbeater.cc:511] Master 127.13.15.190:43289 requested a full tablet report, sending...
I20260929 06:30:05.592419 23639 ts_manager.cc:194] Registered new tserver with Master: 152221b172304bc5bc34ffd7c62aeba9 (127.13.15.130:46685)
I20260929 06:30:05.592898 23639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:60269
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:30:05.613740 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:05.698513 23969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:05.698729 23969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:05.698756 23969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:05.699930 23969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:05.699970 23969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:05.699995 23969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:30:05.701542 23969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:05.701759 23969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:05.701972 23969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:30:05.707391 23969 server_base.cc:1034] running on GCE node
W20260929 06:30:05.707331 23974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.707365 23975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:05.707331 23977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:05.707754 23969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:05.707965 23969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:05.709103 23969 hybrid_clock.cc:648] HybridClock initialized: now 1790663405709058 us; error 63 us; skew 500 ppm
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:05.711681 23969 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:30:05.712149 23969 webserver.cc:460] Webserver started at http://127.13.15.131:43719/ using document root <none> and password file <none>
I20260929 06:30:05.712404 23969 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:05.712450 23969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:05.712623 23969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:30:05.713690 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "3fa1299897ef409ab4a75239b830ff96"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "e7ea20b0eadf3af6f9057233b1535646"
server_key_iv: "ebd6e26094ae9a4ac486718be0bf6429"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.714046 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "3fa1299897ef409ab4a75239b830ff96"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "e7ea20b0eadf3af6f9057233b1535646"
server_key_iv: "ebd6e26094ae9a4ac486718be0bf6429"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.715402 23969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:30:05.716288 23984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.716504 23969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:30:05.716606 23969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "3fa1299897ef409ab4a75239b830ff96"
format_stamp: "Formatted at 2026-09-29 06:30:05 on dist-test-slave-sm0p"
server_key: "e7ea20b0eadf3af6f9057233b1535646"
server_key_iv: "ebd6e26094ae9a4ac486718be0bf6429"
server_key_version: "encryptionkey@0"
I20260929 06:30:05.716699 23969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:05.731941 23969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:05.734560 23969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:05.734727 23969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:05.734982 23969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:05.735311 23969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:05.735366 23969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.735412 23969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:05.735436 23969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:05.741771 23969 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40955
I20260929 06:30:05.741824 24097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40955 every 8 connection(s)
I20260929 06:30:05.742211 23969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:30:05.748766 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 23969
I20260929 06:30:05.748869 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:30:05.749058 13374 external_mini_cluster.cc:1428] Setting key cdc00a9ac0f510dcd32f58199b797c6c
Sep 29 06:30:05 dist-test-slave-sm0p krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:05.754074 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.744215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:46009 (local address 127.13.15.190:43289)
0929 06:30:05.744265 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.744267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.744907 (+ 640us) server_negotiation.cc:407] Connection header received
0929 06:30:05.745717 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.745719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.745747 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.745803 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:05.747025 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.747531 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.748583 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.748798 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.748884 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.751473 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:05.751475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:05.751475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:05.751483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:05.752691 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.753106 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.753107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.753108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.753172 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.753430 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.753431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.753432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.753550 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:05.753630 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.753903 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.753984 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:30:05.754315 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.744201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:43289 (local address 127.13.15.131:46009)
0929 06:30:05.744748 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.744800 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.745584 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.745876 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.745880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.746608 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:05.746929 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.746937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.747633 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.747634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.748471 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.748473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.748936 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.748937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.749068 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.749739 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:05.749769 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:05.751359 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:05.752779 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.752785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.752787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.753010 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.753243 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:05.753244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:05.753245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:05.753357 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:05.753619 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:05.753625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:05.753769 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.753883 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.754039 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":78,"threads_started":1}
I20260929 06:30:05.754791 24098 heartbeater.cc:347] Connected to a master server at 127.13.15.190:43289
I20260929 06:30:05.754894 24098 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:05.755077 24098 heartbeater.cc:511] Master 127.13.15.190:43289 requested a full tablet report, sending...
I20260929 06:30:05.755478 23639 ts_manager.cc:194] Registered new tserver with Master: 3fa1299897ef409ab4a75239b830ff96 (127.13.15.131:40955)
I20260929 06:30:05.755854 23639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:46009
I20260929 06:30:05.760886 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:30:05.767859 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.762237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.13.15.190:43289)
0929 06:30:05.762295 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.762297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.762415 (+ 118us) server_negotiation.cc:407] Connection header received
0929 06:30:05.762492 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.762493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.762502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.762547 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:05.763033 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.763502 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.764181 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.764352 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.764399 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.765480 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:05.765482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:05.765482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:05.765487 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:05.766929 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.767239 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.767240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.767240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.767287 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:05.767453 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:05.767453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:05.767453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:05.767582 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:05.767657 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.767747 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.767800 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:30:05.769372 23639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:30:05.771034 23639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:30:05.777555 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46685 (local address 127.0.0.1:43918)
0929 06:30:05.773464 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.773475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.773524 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.773771 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.773774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.773777 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:05.773858 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.773860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.774959 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.774960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.776235 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.776237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777242 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.777332 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777346 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.777413 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.777480 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260929 06:30:05.777565 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57546 (local address 127.13.15.131:40955)
0929 06:30:05.773791 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.773796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.773865 (+ 69us) server_negotiation.cc:407] Connection header received
0929 06:30:05.773962 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.773965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.773980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.774240 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:05.774414 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.775441 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.776576 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777144 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777214 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777273 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.777453 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.777488 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
I20260929 06:30:05.777698 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.13.15.129:45773)
0929 06:30:05.773883 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.773887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.773895 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:30:05.773941 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.773943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.773958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.774057 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:05.774230 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.775341 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.776628 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777343 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777438 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777491 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.777582 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.777621 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":72,"threads_started":1}
I20260929 06:30:05.777906 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.13.15.130:46685)
0929 06:30:05.773516 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.773521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.773556 (+ 35us) server_negotiation.cc:407] Connection header received
0929 06:30:05.773595 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.773596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.773610 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.773742 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:05.773927 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.774849 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.776401 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777168 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777240 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777292 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.777767 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.777823 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260929 06:30:05.778115 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40955 (local address 127.0.0.1:57546)
0929 06:30:05.773796 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.773806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.773872 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.774307 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.774307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.774309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:05.774340 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.774342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.775522 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.775523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.776479 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.776482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777301 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.777349 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777351 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.777413 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.778052 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":573,"thread_start_us":61,"threads_started":1}
I20260929 06:30:05.777923 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.773184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:45773 (local address 127.0.0.1:51790)
0929 06:30:05.773365 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.773406 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.773470 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.774071 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.774071 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.774073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:05.774112 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.774113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.775456 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.775457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.776538 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.776540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.777441 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.777442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.777506 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.777511 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.777568 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.777608 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260929 06:30:05.778935 24032 tablet_service.cc:1467] Processing CreateTablet for tablet c842532f622d4ba095f6f6406b41a35a (DEFAULT_TABLE table=test-table [id=93f8f587ec2749d49bd5658543d0b453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:05.778939 23898 tablet_service.cc:1467] Processing CreateTablet for tablet c842532f622d4ba095f6f6406b41a35a (DEFAULT_TABLE table=test-table [id=93f8f587ec2749d49bd5658543d0b453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:05.779178 23764 tablet_service.cc:1467] Processing CreateTablet for tablet c842532f622d4ba095f6f6406b41a35a (DEFAULT_TABLE table=test-table [id=93f8f587ec2749d49bd5658543d0b453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:05.779317 23898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c842532f622d4ba095f6f6406b41a35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:05.779349 24032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c842532f622d4ba095f6f6406b41a35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:05.779505 23764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c842532f622d4ba095f6f6406b41a35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:05.781903 24115 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Bootstrap starting.
I20260929 06:30:05.782580 24116 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Bootstrap starting.
I20260929 06:30:05.782801 24115 tablet_bootstrap.cc:654] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:05.783274 24116 tablet_bootstrap.cc:654] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:05.783444 24115 log.cc:824] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:05.783883 24116 log.cc:824] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:05.784274 24115 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: No bootstrap required, opened a new log
I20260929 06:30:05.784250 24117 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Bootstrap starting.
I20260929 06:30:05.784340 24115 ts_tablet_manager.cc:1397] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 06:30:05.784572 24116 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: No bootstrap required, opened a new log
I20260929 06:30:05.784665 24116 ts_tablet_manager.cc:1397] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:30:05.784861 24117 tablet_bootstrap.cc:654] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:05.785382 24117 log.cc:824] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:05.786239 24117 tablet_bootstrap.cc:492] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: No bootstrap required, opened a new log
I20260929 06:30:05.786311 24117 ts_tablet_manager.cc:1397] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:30:05.786617 24116 raft_consensus.cc:348] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.786808 24116 raft_consensus.cc:374] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:05.786684 24115 raft_consensus.cc:348] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.786850 24116 raft_consensus.cc:729] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152221b172304bc5bc34ffd7c62aeba9, State: Initialized, Role: FOLLOWER
I20260929 06:30:05.786856 24115 raft_consensus.cc:374] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:05.786891 24115 raft_consensus.cc:729] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa1299897ef409ab4a75239b830ff96, State: Initialized, Role: FOLLOWER
I20260929 06:30:05.786953 24116 consensus_queue.cc:260] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.787002 24115 consensus_queue.cc:260] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.787222 24116 ts_tablet_manager.cc:1428] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:30:05.787344 23964 heartbeater.cc:503] Master 127.13.15.190:43289 was elected leader, sending a full tablet report...
I20260929 06:30:05.787268 24115 ts_tablet_manager.cc:1428] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:30:05.787671 24098 heartbeater.cc:503] Master 127.13.15.190:43289 was elected leader, sending a full tablet report...
I20260929 06:30:05.788316 24117 raft_consensus.cc:348] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.788465 24117 raft_consensus.cc:374] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:05.788524 24117 raft_consensus.cc:729] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea3abbb641d452b88d263a581a3974b, State: Initialized, Role: FOLLOWER
I20260929 06:30:05.788631 24117 consensus_queue.cc:260] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.788895 24117 ts_tablet_manager.cc:1428] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:30:05.788926 23830 heartbeater.cc:503] Master 127.13.15.190:43289 was elected leader, sending a full tablet report...
W20260929 06:30:05.831459 23965 tablet.cc:2369] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:30:05.892083 24123 raft_consensus.cc:482] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:30:05.892275 24123 raft_consensus.cc:504] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.892668 24123 leader_election.cc:288] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 152221b172304bc5bc34ffd7c62aeba9 (127.13.15.130:46685), 3fa1299897ef409ab4a75239b830ff96 (127.13.15.131:40955)
I20260929 06:30:05.895656 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.892856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46685 (local address 127.13.15.129:49093)
0929 06:30:05.892911 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.892923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.892964 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.893174 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.893175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.893194 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:05.893263 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.893265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.893970 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.893971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.894774 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.894776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.895381 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.895382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.895459 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.895464 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.895525 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.895573 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 06:30:05.895668 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.892929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:49093 (local address 127.13.15.130:46685)
0929 06:30:05.892994 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.892996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.893008 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:30:05.893051 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.893052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.893061 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.893153 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:05.893337 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.893889 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.894870 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.895303 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.895357 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.895364 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.895563 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.895597 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:30:05.895838 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.892962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40955 (local address 127.13.15.129:46949)
0929 06:30:05.893109 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.893118 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:05.893164 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:05.893410 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:05.893411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:05.893413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:05.893478 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.893480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.894185 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.894186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.894978 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:05.894980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.895589 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.895590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:05.895652 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.895656 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:30:05.895723 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 06:30:05.895762 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260929 06:30:05.895871 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.893106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:46949 (local address 127.13.15.131:40955)
0929 06:30:05.893275 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.893277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.893284 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:30:05.893311 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.893312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.893321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.893369 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:05.893554 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.894054 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.895068 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.895511 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.895574 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.895581 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.895749 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.895788 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260929 06:30:05.896214 24052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c842532f622d4ba095f6f6406b41a35a" candidate_uuid: "eea3abbb641d452b88d263a581a3974b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa1299897ef409ab4a75239b830ff96" is_pre_election: true
I20260929 06:30:05.896369 24052 raft_consensus.cc:2454] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eea3abbb641d452b88d263a581a3974b in term 0.
I20260929 06:30:05.896564 23918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c842532f622d4ba095f6f6406b41a35a" candidate_uuid: "eea3abbb641d452b88d263a581a3974b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152221b172304bc5bc34ffd7c62aeba9" is_pre_election: true
I20260929 06:30:05.896592 23718 leader_election.cc:302] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa1299897ef409ab4a75239b830ff96, eea3abbb641d452b88d263a581a3974b; no voters:
I20260929 06:30:05.896664 23918 raft_consensus.cc:2454] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eea3abbb641d452b88d263a581a3974b in term 0.
I20260929 06:30:05.896739 24123 raft_consensus.cc:2781] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:30:05.896790 24123 raft_consensus.cc:482] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:30:05.896819 24123 raft_consensus.cc:3037] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:05.897491 24123 raft_consensus.cc:504] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.897586 24123 leader_election.cc:288] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [CANDIDATE]: Term 1 election: Requested vote from peers 152221b172304bc5bc34ffd7c62aeba9 (127.13.15.130:46685), 3fa1299897ef409ab4a75239b830ff96 (127.13.15.131:40955)
I20260929 06:30:05.897782 23918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c842532f622d4ba095f6f6406b41a35a" candidate_uuid: "eea3abbb641d452b88d263a581a3974b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152221b172304bc5bc34ffd7c62aeba9"
I20260929 06:30:05.897809 24052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c842532f622d4ba095f6f6406b41a35a" candidate_uuid: "eea3abbb641d452b88d263a581a3974b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa1299897ef409ab4a75239b830ff96"
I20260929 06:30:05.897864 23918 raft_consensus.cc:3037] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:05.897866 24052 raft_consensus.cc:3037] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:05.898615 24052 raft_consensus.cc:2454] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eea3abbb641d452b88d263a581a3974b in term 1.
I20260929 06:30:05.898622 23918 raft_consensus.cc:2454] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eea3abbb641d452b88d263a581a3974b in term 1.
I20260929 06:30:05.898801 23718 leader_election.cc:302] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152221b172304bc5bc34ffd7c62aeba9, eea3abbb641d452b88d263a581a3974b; no voters:
I20260929 06:30:05.898903 24123 raft_consensus.cc:2781] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:05.899057 24123 raft_consensus.cc:686] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 1 LEADER]: Becoming Leader. State: Replica: eea3abbb641d452b88d263a581a3974b, State: Running, Role: LEADER
I20260929 06:30:05.899160 24123 consensus_queue.cc:237] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } }
I20260929 06:30:05.900024 23636 catalog_manager.cc:5495] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b reported cstate change: term changed from 0 to 1, leader changed from <none> to eea3abbb641d452b88d263a581a3974b (127.13.15.129). New cstate: current_term: 1 leader_uuid: "eea3abbb641d452b88d263a581a3974b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea3abbb641d452b88d263a581a3974b" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 45773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 } health_report { overall_health: UNKNOWN } } }
W20260929 06:30:05.909154 23831 tablet.cc:2369] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:30:05.924552 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.922324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.13.15.129:45773)
0929 06:30:05.922390 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:05.922392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:05.922400 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:30:05.922454 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:05.922455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:05.922472 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:05.922535 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:30:05.922669 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.923125 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.923859 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:05.924030 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:05.924079 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:05.924212 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:30:05.924312 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:30:05.924371 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:05.924405 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 06:30:05.924423 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:30:05.926352 23918 raft_consensus.cc:1261] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 1 FOLLOWER]: Refusing update from remote peer eea3abbb641d452b88d263a581a3974b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:30:05.926357 24052 raft_consensus.cc:1261] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 1 FOLLOWER]: Refusing update from remote peer eea3abbb641d452b88d263a581a3974b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:30:05.926666 24125 consensus_queue.cc:1035] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa1299897ef409ab4a75239b830ff96" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:30:05.926839 24123 consensus_queue.cc:1035] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [LEADER]: Connected to new peer: Peer: permanent_uuid: "152221b172304bc5bc34ffd7c62aeba9" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:30:05.929421 24130 mvcc.cc:205] Tried to move back new op lower bound from 7334557310671613952 to 7334557310563749888. Current Snapshot: MvccSnapshot[applied={T|T < 7334557310671613952}]
I20260929 06:30:05.931813 23636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:30:05.931921 23636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:30:05.932703 23636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd89c7c72cef46f4a12ceb5ddb0fc6ed: Sending DeleteTablet for 3 replicas of tablet c842532f622d4ba095f6f6406b41a35a
I20260929 06:30:05.933048 23764 tablet_service.cc:1514] Processing DeleteTablet for tablet c842532f622d4ba095f6f6406b41a35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:51790
I20260929 06:30:05.933151 23898 tablet_service.cc:1514] Processing DeleteTablet for tablet c842532f622d4ba095f6f6406b41a35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:43918
I20260929 06:30:05.933271 24138 tablet_replica.cc:330] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: stopping tablet replica
I20260929 06:30:05.933362 24138 raft_consensus.cc:2229] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:30:05.933418 24032 tablet_service.cc:1514] Processing DeleteTablet for tablet c842532f622d4ba095f6f6406b41a35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:57546
I20260929 06:30:05.933481 24138 raft_consensus.cc:2258] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:30:05.933604 24140 tablet_replica.cc:330] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: stopping tablet replica
I20260929 06:30:05.933696 24140 raft_consensus.cc:2229] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:30:05.933691 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23701
I20260929 06:30:05.933929 24138 ts_tablet_manager.cc:1905] T c842532f622d4ba095f6f6406b41a35a P eea3abbb641d452b88d263a581a3974b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:30:05.934425 24140 raft_consensus.cc:2258] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:30:05.934877 24140 ts_tablet_manager.cc:1905] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:30:05.935240 24139 tablet_replica.cc:330] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: stopping tablet replica
I20260929 06:30:05.935364 24139 raft_consensus.cc:2229] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:30:05.935490 24139 raft_consensus.cc:2258] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:30:05.935863 24139 ts_tablet_manager.cc:1905] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:30:05.936841 24140 ts_tablet_manager.cc:1918] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:30:05.936920 24140 log.cc:1196] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/c842532f622d4ba095f6f6406b41a35a
I20260929 06:30:05.937072 24139 ts_tablet_manager.cc:1918] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:30:05.937160 24139 log.cc:1196] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/c842532f622d4ba095f6f6406b41a35a
I20260929 06:30:05.937239 24140 ts_tablet_manager.cc:1939] T c842532f622d4ba095f6f6406b41a35a P 3fa1299897ef409ab4a75239b830ff96: Deleting consensus metadata
I20260929 06:30:05.937459 24139 ts_tablet_manager.cc:1939] T c842532f622d4ba095f6f6406b41a35a P 152221b172304bc5bc34ffd7c62aeba9: Deleting consensus metadata
I20260929 06:30:05.937665 23624 catalog_manager.cc:4841] TS 3fa1299897ef409ab4a75239b830ff96 (127.13.15.131:40955): tablet c842532f622d4ba095f6f6406b41a35a (table test-table [id=93f8f587ec2749d49bd5658543d0b453]) successfully deleted
I20260929 06:30:05.937917 23624 catalog_manager.cc:4841] TS 152221b172304bc5bc34ffd7c62aeba9 (127.13.15.130:46685): tablet c842532f622d4ba095f6f6406b41a35a (table test-table [id=93f8f587ec2749d49bd5658543d0b453]) successfully deleted
W20260929 06:30:05.940189 23623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:45773 (error 108)
I20260929 06:30:05.940479 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23835
I20260929 06:30:05.941147 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:05.940654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:45773 (local address 127.0.0.1:51806)
0929 06:30:05.941063 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:05.941106 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:45773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389}
W20260929 06:30:05.942526 23623 catalog_manager.cc:4584] TS eea3abbb641d452b88d263a581a3974b (127.13.15.129:45773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c842532f622d4ba095f6f6406b41a35a: Network error: Client connection negotiation failed: client connection to 127.13.15.129:45773: connect: Connection refused (error 111)
I20260929 06:30:05.945546 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23969
I20260929 06:30:05.949663 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 23607
2026-09-29T06:30:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2015 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:30:06 dist-test-slave-sm0p krb5kdc[24144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:30:06 dist-test-slave-sm0p krb5kdc[24144](info): set up 2 sockets
Sep 29 06:30:06 dist-test-slave-sm0p krb5kdc[24144](info): commencing operation
krb5kdc: starting...
W20260929 06:30:06.986425 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:30:07 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:30:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangeradmin_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangeradmin_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangerlookup_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangerlookup_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
I20260929 06:30:07.152863 13374 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJYAK93/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/postgres -l logfile start
2026-09-29 06:30:09.806 UTC [24174] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 06:30:09.806 UTC [24174] LOG: listening on IPv4 address "127.13.15.148", port 40187
2026-09-29 06:30:09.810 UTC [24174] LOG: listening on Unix socket "/tmp/.s.PGSQL.40187"
2026-09-29 06:30:09.831 UTC [24177] LOG: database system was shut down at 2026-09-29 06:30:08 UTC
2026-09-29 06:30:09.835 UTC [24174] LOG: database system is ready to accept connections
I20260929 06:30:10.802608 13374 mini_postgres.cc:96] Postgres bound to 40187
2026-09-29 06:30:10.808 UTC [24185] FATAL: database "slave" does not exist
127.13.15.148:40187 - accepting connections
I20260929 06:30:10.809223 13374 mini_ranger.cc:161] Starting Ranger...
I20260929 06:30:10.824324 13374 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 06:30:11.351625 13374 mini_ranger.cc:87] Created ranger Postgres database
I20260929 06:30:11.351712 13374 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin
2026-09-29 06:30:11,629 [I] DB FLAVOR :POSTGRES
2026-09-29 06:30:11,630 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 06:30:11,630 [I] Checking connection..
2026-09-29 06:30:11,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 06:30:11,902 [I] Checking connection passed.
2026-09-29 06:30:11,903 [I] --------- Verifying version history table ---------
2026-09-29 06:30:11,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:12,142 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 06:30:12,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:12,394 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 06:30:12,395 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 06:30:12,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 06:30:12.633 UTC [24249] WARNING: there is no transaction in progress
2026-09-29 06:30:12,639 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 06:30:12,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:12,878 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 06:30:12,878 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 06:30:12,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 06:30:13,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 06:30:13,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sm0p.c.gcp-upstream.internal','N') ;"
2026-09-29 06:30:13,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 06:30:13,911 [I] Table x_portal_user does not exist in database ranger
2026-09-29 06:30:13,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 06:30:14,165 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 06:30:14,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 06:30:14,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 06:30:14,686 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 06:30:14,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 06:30:15.677 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.695 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.714 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.731 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.748 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.765 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.825 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.833 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.848 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.858 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.869 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.884 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.895 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.902 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.910 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:15.917 UTC [24357] WARNING: there is no transaction in progress
2026-09-29 06:30:16,749 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 06:30:16,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 06:30:17,012 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 06:30:17,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 06:30:17,250 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 06:30:17,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 06:30:17,501 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 06:30:17,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 06:30:17,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 06:30:18,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sm0p.c.gcp-upstream.internal';"
2026-09-29 06:30:18,262 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 06:30:18,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 06:30:18,494 [I] DB_PATCHES have already been applied
I20260929 06:30:18.499897 13374 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJYAK93/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 06:30:18.499994 13374 mini_ranger.cc:193] Using host: 127.13.15.148
I20260929 06:30:18.503233 13374 mini_ranger.cc:227] Ranger admin URL: http://127.13.15.148:60055
Sep 29, 2026 6:30:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:30:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:30:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 6:30:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 6:30:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 6:30:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 6:30:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 06:30:19.616467 13374 mini_ranger.cc:160] Time spent starting Ranger: real 8.807s user 0.000s sys 0.009s
Sep 29, 2026 6:30:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 6:30:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 6:30:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 6:30:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 6:30:40 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 06:30:41.053495 13374 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangerkms_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/rangerkms_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.15.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.15.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 06:30:41 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 06:30:41.138404 13374 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 06:30:41.147001 13374 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 06:30:42.025873 13374 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 06:30:42.025957 13374 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms
I20260929 06:30:42.025975 13374 mini_ranger_kms.cc:227] Using postgres at 127.13.15.148:40187
2026-09-29 06:30:42,188 [I] DB FLAVOR :POSTGRES
2026-09-29 06:30:42,189 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 06:30:42,189 [I] Checking connection
2026-09-29 06:30:42,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 06:30:42,432 [I] connection success
2026-09-29 06:30:42,432 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 06:30:42,432 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 06:30:42,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 06:30:42,661 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 06:30:42,662 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 06:30:42,662 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 06:30:42,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:40187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 06:30:42,917 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 06:30:43.347285 13374 mini_ranger_kms.cc:310] Created kms service
I20260929 06:30:43.516436 13374 mini_ranger_kms.cc:326] Created kudu user
I20260929 06:30:43.563289 13374 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 06:30:43 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:43.939906 13374 mini_ranger_kms.cc:384] Added ranger policy
I20260929 06:30:43.940039 13374 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 06:30:43.940065 13374 mini_ranger_kms.cc:240] Using host: 127.13.15.148
I20260929 06:30:43.944145 13374 mini_ranger_kms.cc:276] Ranger KMS PID: 24537
I20260929 06:30:43.944280 13374 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.15.148:46371
Sep 29, 2026 6:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 6:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 6:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 6:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 06:30:46.172397 13374 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.034s user 0.000s sys 0.146s
I20260929 06:30:46.172519 13374 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 06:30:46.172598 13374 mini_ranger_kms.cc:401] 127.13.15.148:46371/kms/v1/keys
Sep 29 06:30:46 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790663446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 06:30:46 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
Sep 29 06:30:47 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
Sep 29 06:30:49 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:30:49.404610 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44925
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44925
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:46371/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:49.484695 24588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:49.484906 24588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:49.484930 24588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:49.486091 24588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:30:49.486179 24588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:49.486231 24588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:49.486250 24588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:30:49.486276 24588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:30:49.487816 24588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.15.148:46371/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44925
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:49.488047 24588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:49.488281 24588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:49.493278 24593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:49.493294 24594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:49.493278 24596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:49.493402 24588 server_base.cc:1034] running on GCE node
I20260929 06:30:49.493767 24588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:49.494035 24588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:49.495203 24588 hybrid_clock.cc:648] HybridClock initialized: now 1790663449495193 us; error 46 us; skew 500 ppm
Sep 29 06:30:49 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663449, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:49.497835 24588 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:30:49.498358 24588 webserver.cc:460] Webserver started at http://127.13.15.190:41021/ using document root <none> and password file <none>
I20260929 06:30:49.498567 24588 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:49.498621 24588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:49.498755 24588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:30:49 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663449, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:49.552134 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706"
format_stamp: "Formatted at 2026-09-29 06:30:49 on dist-test-slave-sm0p"
server_key: "da71261aff6ce60cfde76c208645b458"
server_key_iv: "fba7ccc231871d938b2ef10d9467ad85"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:49.552553 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706"
format_stamp: "Formatted at 2026-09-29 06:30:49 on dist-test-slave-sm0p"
server_key: "da71261aff6ce60cfde76c208645b458"
server_key_iv: "fba7ccc231871d938b2ef10d9467ad85"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:49.553970 24588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260929 06:30:49.568845 24603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:49.569089 24588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:30:49.569192 24588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706"
format_stamp: "Formatted at 2026-09-29 06:30:49 on dist-test-slave-sm0p"
server_key: "da71261aff6ce60cfde76c208645b458"
server_key_iv: "fba7ccc231871d938b2ef10d9467ad85"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:49.569311 24588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:49.589054 24588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:49.591125 24588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:49.591276 24588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:49.596163 24588 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44925
I20260929 06:30:49.596189 24655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44925 every 8 connection(s)
I20260929 06:30:49.596547 24588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:30:49.597204 24656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:49.599571 24656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: Bootstrap starting.
I20260929 06:30:49.600131 24656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:49.600389 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 24588
I20260929 06:30:49.600502 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:30:49.600485 24656 log.cc:824] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:49.600729 13374 external_mini_cluster.cc:1428] Setting key f05b0c30d546cc26d7cd460aac6f9e72
I20260929 06:30:49.601262 24656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: No bootstrap required, opened a new log
I20260929 06:30:49.604101 24656 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } }
I20260929 06:30:49.604269 24656 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:49.604298 24656 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fd9390c9c84ee2b8f258e1cbdc8706, State: Initialized, Role: FOLLOWER
I20260929 06:30:49.604462 24656 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } }
I20260929 06:30:49.604539 24656 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:30:49.604566 24656 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:30:49.604629 24656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:49.605290 24656 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } }
I20260929 06:30:49.605396 24656 leader_election.cc:302] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0fd9390c9c84ee2b8f258e1cbdc8706; no voters:
I20260929 06:30:49.605576 24656 leader_election.cc:288] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:30:49.605625 24661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:49.605830 24661 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [term 1 LEADER]: Becoming Leader. State: Replica: c0fd9390c9c84ee2b8f258e1cbdc8706, State: Running, Role: LEADER
I20260929 06:30:49.605877 24656 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:30:49.605986 24661 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } }
I20260929 06:30:49.606726 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } } }
I20260929 06:30:49.606797 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:49.606915 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0fd9390c9c84ee2b8f258e1cbdc8706. Latest consensus state: current_term: 1 leader_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fd9390c9c84ee2b8f258e1cbdc8706" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44925 } } }
I20260929 06:30:49.606971 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706 [sys.catalog]: This master's current role is: LEADER
I20260929 06:30:49.607286 24656 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 29 06:30:49 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:49.610338 24671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:30:49.611279 24671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:30:49.615082 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:49.601239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.13.15.190:44925)
0929 06:30:49.601457 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:49.601468 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:49.601502 (+ 34us) server_negotiation.cc:407] Connection header received
0929 06:30:49.602589 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:49.602596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:49.602644 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:49.602783 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:49.603379 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:49.604255 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:49.605130 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:49.605678 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:49.606106 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:49.609435 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:49.609508 (+ 73us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:49.609509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:49.609638 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:49.612853 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:49.613284 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:49.613287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:49.613288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:49.613399 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:49.613657 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:49.613659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:49.613659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:49.614000 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:49.614597 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:49.614674 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 06:30:49.614822 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260929 06:30:49.618594 24671 catalog_manager.cc:1347] Generated new cluster ID: 4bc57e9182084d5e82d3e2779468a44b
I20260929 06:30:49.618644 24671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:30:49.654748 24671 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:30:49.655613 24671 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:30:49.670734 24671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: Generated new TSK 0
I20260929 06:30:49.671054 24671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:30:50.330742 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:50.430089 24701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:50.430362 24701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:50.430398 24701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:50.432086 24701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:50.432129 24701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:50.432156 24701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:30:50.433631 24701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.15.148:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:50.433851 24701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:50.434087 24701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:50.439940 24706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.439971 24707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.440191 24709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:50.440481 24701 server_base.cc:1034] running on GCE node
I20260929 06:30:50.440706 24701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:50.441011 24701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:50.442194 24701 hybrid_clock.cc:648] HybridClock initialized: now 1790663450442174 us; error 35 us; skew 500 ppm
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:50.445576 24701 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:30:50.446216 24701 webserver.cc:460] Webserver started at http://127.13.15.129:45071/ using document root <none> and password file <none>
I20260929 06:30:50.446480 24701 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:50.446527 24701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:50.446671 24701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:50.462548 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "f58d8846a30845778b5b2ac559012ac7"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "a7297a2a359580d1646b0722d18cc0e8"
server_key_iv: "623a813d3f3b0e17d35eef4c2973ed72"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.463106 24701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "f58d8846a30845778b5b2ac559012ac7"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "a7297a2a359580d1646b0722d18cc0e8"
server_key_iv: "623a813d3f3b0e17d35eef4c2973ed72"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.467259 24701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260929 06:30:50.483144 24718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.483363 24701 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260929 06:30:50.483451 24701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "f58d8846a30845778b5b2ac559012ac7"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "a7297a2a359580d1646b0722d18cc0e8"
server_key_iv: "623a813d3f3b0e17d35eef4c2973ed72"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.483541 24701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:50.524319 24701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:50.526424 24701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:50.526576 24701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:50.526816 24701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:50.527195 24701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:50.527245 24701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.527310 24701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:50.527354 24701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.533604 24701 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:35919
I20260929 06:30:50.534109 24701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:30:50.537184 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 24701
I20260929 06:30:50.537298 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:30:50.537528 13374 external_mini_cluster.cc:1428] Setting key 8d0350001fbfaafb4e412d08fba6eac2
I20260929 06:30:50.539096 24831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:35919 every 8 connection(s)
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:50.550971 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:50.539803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44925 (local address 127.13.15.129:43049)
0929 06:30:50.539990 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:50.540051 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:50.540655 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:50.541073 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:50.541078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:50.541328 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:50.541604 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:50.541610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.542669 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.542670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:50.543959 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:50.543962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.544533 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.544534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:50.544684 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:50.545262 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:50.545273 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:50.547052 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:50.549258 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:50.549264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:50.549267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:50.549583 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:50.549814 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:50.549814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:50.549815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:50.549949 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:50.550310 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:50.550317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:50.550472 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 06:30:50.550586 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 06:30:50.550734 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260929 06:30:50.551514 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:50.539826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:43049 (local address 127.13.15.190:44925)
0929 06:30:50.540839 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:50.540841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:50.540848 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:30:50.540870 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:50.540871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:50.540882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:50.540952 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:50.541720 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.542560 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.544103 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.544400 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.544469 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:50.547153 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:50.547154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:50.547154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:50.547161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:50.549163 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:50.549672 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:50.549673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:50.549674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:50.549741 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:50.550039 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:50.550040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:50.550040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:50.550217 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:50.550283 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:50.551326 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0929 06:30:50.551415 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":83,"threads_started":1}
I20260929 06:30:50.552260 24832 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44925
I20260929 06:30:50.552354 24832 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:50.552537 24832 heartbeater.cc:511] Master 127.13.15.190:44925 requested a full tablet report, sending...
I20260929 06:30:50.553200 24620 ts_manager.cc:194] Registered new tserver with Master: f58d8846a30845778b5b2ac559012ac7 (127.13.15.129:35919)
I20260929 06:30:50.554072 24620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:43049
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:30:50.584842 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 06:30:50.703361 24840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:50.703621 24840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:50.703655 24840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:50.705303 24840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:50.705384 24840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:50.705469 24840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:30:50.707647 24840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.15.148:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:50.707871 24840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:50.708150 24840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:50.714598 24848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.714596 24846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.714804 24845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:50.716248 24840 server_base.cc:1034] running on GCE node
I20260929 06:30:50.716450 24840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:50.716694 24840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:50.717864 24840 hybrid_clock.cc:648] HybridClock initialized: now 1790663450717826 us; error 54 us; skew 500 ppm
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:50.721087 24840 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:30:50.721608 24840 webserver.cc:460] Webserver started at http://127.13.15.130:38541/ using document root <none> and password file <none>
I20260929 06:30:50.721865 24840 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:50.721930 24840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:50.722056 24840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:50.738627 24840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "72712f07e7c842db975ebd52c521c8b8"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "771784d07241ac5beba4ccbdbe50b3aa"
server_key_iv: "b764d73d613e2f32c29fbd007b5c99cf"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.739101 24840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "72712f07e7c842db975ebd52c521c8b8"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "771784d07241ac5beba4ccbdbe50b3aa"
server_key_iv: "b764d73d613e2f32c29fbd007b5c99cf"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.740752 24840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:30:50.754375 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.754607 24840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:30:50.754698 24840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "72712f07e7c842db975ebd52c521c8b8"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "771784d07241ac5beba4ccbdbe50b3aa"
server_key_iv: "b764d73d613e2f32c29fbd007b5c99cf"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:50.754783 24840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:50.810477 24840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:50.813148 24840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:50.813371 24840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:50.813644 24840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:50.814011 24840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:50.814096 24840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.814174 24840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:50.814221 24840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:50.821545 24840 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:36629
I20260929 06:30:50.821821 24969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:36629 every 8 connection(s)
I20260929 06:30:50.822022 24840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:30:50.825806 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 24840
I20260929 06:30:50.825912 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:30:50.826097 13374 external_mini_cluster.cc:1428] Setting key 5d3daefa586b8671c18ee697947a9980
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:50.833386 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:50.822796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:37323 (local address 127.13.15.190:44925)
0929 06:30:50.822861 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:50.822863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:50.823090 (+ 227us) server_negotiation.cc:407] Connection header received
0929 06:30:50.823772 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:50.823775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:50.823793 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:50.823832 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:50.824749 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.825523 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.827080 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.827379 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.827426 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:50.830057 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:50.830059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:50.830059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:50.830067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:50.831703 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:50.832213 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:50.832214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:50.832215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:50.832276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:50.832571 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:50.832572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:50.832573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:50.832701 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:50.832747 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:50.833202 (+ 455us) server_negotiation.cc:299] Negotiation successful
0929 06:30:50.833289 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:30:50.833870 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:50.822770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44925 (local address 127.13.15.130:37323)
0929 06:30:50.822944 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:50.822993 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:50.823641 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:50.824067 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:50.824073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:50.824353 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:50.824633 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:50.824639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.825640 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.825641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:50.826955 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:50.826958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:50.827521 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:50.827522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:50.827679 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:50.828275 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:50.828285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:50.829931 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:50.831805 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:50.831811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:50.831813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:50.832101 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:50.832354 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:50.832355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:50.832355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:50.832487 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:50.832799 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:50.832805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:50.833020 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 06:30:50.833485 (+ 465us) client_negotiation.cc:240] Negotiation successful
0929 06:30:50.833642 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260929 06:30:50.834546 24970 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44925
I20260929 06:30:50.834649 24970 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:50.834829 24970 heartbeater.cc:511] Master 127.13.15.190:44925 requested a full tablet report, sending...
I20260929 06:30:50.835222 24620 ts_manager.cc:194] Registered new tserver with Master: 72712f07e7c842db975ebd52c521c8b8 (127.13.15.130:36629)
I20260929 06:30:50.835741 24620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:37323
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:30:50.860961 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:30:50.969515 24975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:30:50.969842 24975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:30:50.969894 24975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:30:50.971613 24975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:30:50.971701 24975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:30:50.971758 24975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:30:50.973907 24975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.15.148:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:30:50.974180 24975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:30:50.974498 24975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:30:50.980974 24983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.981102 24981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:30:50.980979 24980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:30:50.981706 24975 server_base.cc:1034] running on GCE node
I20260929 06:30:50.981930 24975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:30:50.982242 24975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:30:50.983405 24975 hybrid_clock.cc:648] HybridClock initialized: now 1790663450983379 us; error 40 us; skew 500 ppm
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:30:50.986671 24975 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:30:50.987205 24975 webserver.cc:460] Webserver started at http://127.13.15.131:45873/ using document root <none> and password file <none>
I20260929 06:30:50.987479 24975 fs_manager.cc:359] Metadata directory not provided
I20260929 06:30:50.987550 24975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:30:50.987749 24975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:30:50 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:51.001008 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "55f551c7530940d5a625b0533d9b1af1"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "3443bb7fcbb43a6c615e1499fdb09deb"
server_key_iv: "d74814e302e117b03c50a645b2eb2726"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:51.001523 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "55f551c7530940d5a625b0533d9b1af1"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "3443bb7fcbb43a6c615e1499fdb09deb"
server_key_iv: "d74814e302e117b03c50a645b2eb2726"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:51.008967 24975 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260929 06:30:51.026262 24990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:51.026469 24975 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260929 06:30:51.026572 24975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "55f551c7530940d5a625b0533d9b1af1"
format_stamp: "Formatted at 2026-09-29 06:30:50 on dist-test-slave-sm0p"
server_key: "3443bb7fcbb43a6c615e1499fdb09deb"
server_key_iv: "d74814e302e117b03c50a645b2eb2726"
server_key_version: "kuduclusterkey@0"
I20260929 06:30:51.026666 24975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:30:51.051687 24975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:30:51.054216 24975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:30:51.054368 24975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:30:51.054628 24975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:30:51.055012 24975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:30:51.055058 24975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:51.055094 24975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:30:51.055162 24975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:30:51.062014 24975 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:33395
I20260929 06:30:51.062454 24975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:30:51 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:30:51.070111 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 24975
I20260929 06:30:51.070262 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:30:51.070482 13374 external_mini_cluster.cc:1428] Setting key 1e699155e19e10464b743eb3d79ab7c1
I20260929 06:30:51.073604 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.063280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:51669 (local address 127.13.15.190:44925)
0929 06:30:51.063532 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.063535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.063612 (+ 77us) server_negotiation.cc:407] Connection header received
0929 06:30:51.064215 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.064216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.064229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.064318 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:51.064985 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.065762 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.067188 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.067486 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.067571 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.070208 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:51.070210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:51.070210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:51.070217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:51.071870 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:51.072418 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:51.072419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:51.072419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:51.072478 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:51.072775 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:51.072776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:51.072777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:51.072910 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:51.072977 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.073403 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.073505 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260929 06:30:51.074304 25103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:33395 every 8 connection(s)
I20260929 06:30:51.074335 25106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.063302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44925 (local address 127.13.15.131:51669)
0929 06:30:51.063458 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.063501 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.064102 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.064391 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.064396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.064628 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:30:51.064888 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.064893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.065859 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.065860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.067094 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.067097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.067584 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.067584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.067732 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.068357 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:30:51.068366 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:30:51.070043 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:30:51.071983 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:51.071989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:51.071991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:51.072322 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:51.072553 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:30:51.072554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:30:51.072554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:30:51.072699 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:30:51.072989 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:30:51.072995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:30:51.073228 (+ 233us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.073901 (+ 673us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.074069 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260929 06:30:51.076225 25104 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44925
I20260929 06:30:51.076375 25104 heartbeater.cc:464] Registering TS with master...
I20260929 06:30:51.076624 25104 heartbeater.cc:511] Master 127.13.15.190:44925 requested a full tablet report, sending...
I20260929 06:30:51.077124 24620 ts_manager.cc:194] Registered new tserver with Master: 55f551c7530940d5a625b0533d9b1af1 (127.13.15.131:33395)
I20260929 06:30:51.077618 24620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:51669
I20260929 06:30:51.082078 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:30:51.097904 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.086531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.13.15.190:44925)
0929 06:30:51.086581 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.086583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.086764 (+ 181us) server_negotiation.cc:407] Connection header received
0929 06:30:51.086870 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.086872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.086884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.086914 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:30:51.088847 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.089576 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.092486 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.092770 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.092816 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.094690 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:30:51.094692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:30:51.094692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:30:51.094699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:30:51.096411 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:51.096819 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:51.096820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:51.096821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:51.096881 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:30:51.097265 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:30:51.097266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:30:51.097266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:30:51.097400 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:30:51.097495 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.097722 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.097807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:30:51.100401 24620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:30:51.102679 24620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 29 06:30:51 dist-test-slave-sm0p krb5kdc[24144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:30:51.111028 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.105748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:33395 (local address 127.0.0.1:39050)
0929 06:30:51.106394 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.106402 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.106434 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.106885 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.106888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.106890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:51.106962 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.106964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.108245 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.108246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.109621 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.109624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.110736 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.110737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.110816 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.110829 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.110898 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.110950 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":65,"threads_started":1}
I20260929 06:30:51.111564 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.105857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.13.15.131:33395)
0929 06:30:51.106653 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.106659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.106668 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:30:51.106709 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.106711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.106726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.106795 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:51.107046 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.108145 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.109770 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.110558 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.110628 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.110701 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.111428 (+ 727us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.111471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":78,"threads_started":1}
I20260929 06:30:51.112358 25038 tablet_service.cc:1467] Processing CreateTablet for tablet 8213eb01855b4644905ce5bb713c7b60 (DEFAULT_TABLE table=test-table [id=f119792567974286814326e5af2b38db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:51.112740 25038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8213eb01855b4644905ce5bb713c7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:51.122241 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.105543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:36629 (local address 127.0.0.1:50090)
0929 06:30:51.106189 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.106230 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.106294 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.118436 (+ 12142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.118438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.118440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:51.118488 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.118490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.119841 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.119842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.120953 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.120956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.121961 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.121962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.122026 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.122044 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.122075 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.122167 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":66,"threads_started":1}
I20260929 06:30:51.122630 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.105689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.13.15.130:36629)
0929 06:30:51.118240 (+ 12551us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.118245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.118257 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:30:51.118296 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.118298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.118315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.118562 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:51.118635 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.119715 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.121102 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.121858 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.122411 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.122487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.122512 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.122549 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12506,"thread_start_us":85,"threads_started":1}
I20260929 06:30:51.123428 24895 tablet_service.cc:1467] Processing CreateTablet for tablet 8213eb01855b4644905ce5bb713c7b60 (DEFAULT_TABLE table=test-table [id=f119792567974286814326e5af2b38db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:51.123797 24895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8213eb01855b4644905ce5bb713c7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:51.126885 25123 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Bootstrap starting.
I20260929 06:30:51.127395 25124 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Bootstrap starting.
I20260929 06:30:51.128002 25123 tablet_bootstrap.cc:654] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:51.129170 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.105913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35919 (local address 127.0.0.1:45582)
0929 06:30:51.106498 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.106505 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.106547 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.123377 (+ 16830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.123378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.123380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:51.123420 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.123421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.124814 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.124815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.125911 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.125914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.128974 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.128975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.129032 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.129037 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.129064 (+ 27us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.129102 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":61,"threads_started":1}
I20260929 06:30:51.129691 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.106113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.13.15.129:35919)
0929 06:30:51.123199 (+ 17086us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.123204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.123215 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:30:51.123248 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.123250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.123267 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.123494 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:51.123573 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.124711 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.126103 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.128884 (+ 2781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.129474 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.129556 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.129581 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.129620 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17049,"thread_start_us":123,"threads_started":1}
I20260929 06:30:51.130404 24766 tablet_service.cc:1467] Processing CreateTablet for tablet 8213eb01855b4644905ce5bb713c7b60 (DEFAULT_TABLE table=test-table [id=f119792567974286814326e5af2b38db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:30:51.130759 24766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8213eb01855b4644905ce5bb713c7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:30:51.131268 25124 tablet_bootstrap.cc:654] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:51.131626 25124 log.cc:824] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:51.133924 25126 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: Bootstrap starting.
I20260929 06:30:51.134435 25123 log.cc:824] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:51.134606 25126 tablet_bootstrap.cc:654] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: Neither blocks nor log segments found. Creating new log.
I20260929 06:30:51.134964 25126 log.cc:824] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: Log is configured to *not* fsync() on all Append() calls
I20260929 06:30:51.135205 25123 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: No bootstrap required, opened a new log
I20260929 06:30:51.135272 25123 ts_tablet_manager.cc:1397] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260929 06:30:51.137108 25123 raft_consensus.cc:348] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.137267 25123 raft_consensus.cc:374] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:51.137295 25123 raft_consensus.cc:729] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f551c7530940d5a625b0533d9b1af1, State: Initialized, Role: FOLLOWER
I20260929 06:30:51.137400 25123 consensus_queue.cc:260] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.137650 25123 ts_tablet_manager.cc:1428] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:30:51.137708 25104 heartbeater.cc:503] Master 127.13.15.190:44925 was elected leader, sending a full tablet report...
I20260929 06:30:51.138417 25126 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: No bootstrap required, opened a new log
I20260929 06:30:51.138515 25126 ts_tablet_manager.cc:1397] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260929 06:30:51.140821 25126 raft_consensus.cc:348] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.140974 25126 raft_consensus.cc:374] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:51.141002 25126 raft_consensus.cc:729] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58d8846a30845778b5b2ac559012ac7, State: Initialized, Role: FOLLOWER
I20260929 06:30:51.141119 25126 consensus_queue.cc:260] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.141348 25126 ts_tablet_manager.cc:1428] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 06:30:51.141382 24832 heartbeater.cc:503] Master 127.13.15.190:44925 was elected leader, sending a full tablet report...
I20260929 06:30:51.142510 25124 tablet_bootstrap.cc:492] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: No bootstrap required, opened a new log
I20260929 06:30:51.142638 25124 ts_tablet_manager.cc:1397] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260929 06:30:51.144512 25124 raft_consensus.cc:348] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.144737 25124 raft_consensus.cc:374] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:30:51.144809 25124 raft_consensus.cc:729] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72712f07e7c842db975ebd52c521c8b8, State: Initialized, Role: FOLLOWER
I20260929 06:30:51.144932 25124 consensus_queue.cc:260] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.145213 25124 ts_tablet_manager.cc:1428] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 06:30:51.145246 24970 heartbeater.cc:503] Master 127.13.15.190:44925 was elected leader, sending a full tablet report...
I20260929 06:30:51.219834 25130 raft_consensus.cc:482] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:30:51.220031 25130 raft_consensus.cc:504] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.220417 25130 leader_election.cc:288] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72712f07e7c842db975ebd52c521c8b8 (127.13.15.130:36629), 55f551c7530940d5a625b0533d9b1af1 (127.13.15.131:33395)
I20260929 06:30:51.224541 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.220953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:39099 (local address 127.13.15.131:33395)
0929 06:30:51.220995 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.220996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.221102 (+ 106us) server_negotiation.cc:407] Connection header received
0929 06:30:51.221211 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.221212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.221224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.221252 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:51.221457 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.222191 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.223627 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.224173 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.224217 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.224225 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.224417 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.224457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:30:51.224931 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.220756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:49759 (local address 127.13.15.130:36629)
0929 06:30:51.220822 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.220825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.220947 (+ 122us) server_negotiation.cc:407] Connection header received
0929 06:30:51.221099 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.221100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.221111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.221181 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:30:51.221350 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.222158 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.224002 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.224520 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.224580 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.224588 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.224825 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.224866 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:30:51.225028 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.220879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:33395 (local address 127.13.15.129:39099)
0929 06:30:51.221032 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.221044 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.221135 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.221314 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.221315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.221316 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:51.221385 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.221387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.222290 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.222291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.223522 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.223525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.224278 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.224278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.224344 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.224347 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.224881 (+ 534us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.224923 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":19,"thread_start_us":77,"threads_started":1}
I20260929 06:30:51.225351 25058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8213eb01855b4644905ce5bb713c7b60" candidate_uuid: "f58d8846a30845778b5b2ac559012ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f551c7530940d5a625b0533d9b1af1" is_pre_election: true
I20260929 06:30:51.225492 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.220644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:36629 (local address 127.13.15.129:49759)
0929 06:30:51.220862 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.220874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:30:51.220977 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:30:51.221198 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:30:51.221198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:30:51.221200 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:30:51.221285 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.221286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.222290 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.222291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.223911 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:30:51.223913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.224647 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.224648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:30:51.224742 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.224745 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:30:51.225381 (+ 636us) client_negotiation.cc:240] Negotiation successful
0929 06:30:51.225422 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":440,"thread_start_us":123,"threads_started":1}
I20260929 06:30:51.225524 25058 raft_consensus.cc:2454] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58d8846a30845778b5b2ac559012ac7 in term 0.
I20260929 06:30:51.225768 24721 leader_election.cc:302] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f551c7530940d5a625b0533d9b1af1, f58d8846a30845778b5b2ac559012ac7; no voters:
I20260929 06:30:51.225934 25130 raft_consensus.cc:2781] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:30:51.225982 25130 raft_consensus.cc:482] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:30:51.226006 25130 raft_consensus.cc:3037] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:51.226397 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8213eb01855b4644905ce5bb713c7b60" candidate_uuid: "f58d8846a30845778b5b2ac559012ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72712f07e7c842db975ebd52c521c8b8" is_pre_election: true
I20260929 06:30:51.226545 24919 raft_consensus.cc:2454] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58d8846a30845778b5b2ac559012ac7 in term 0.
I20260929 06:30:51.226979 25130 raft_consensus.cc:504] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.227118 25130 leader_election.cc:288] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [CANDIDATE]: Term 1 election: Requested vote from peers 72712f07e7c842db975ebd52c521c8b8 (127.13.15.130:36629), 55f551c7530940d5a625b0533d9b1af1 (127.13.15.131:33395)
I20260929 06:30:51.227308 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8213eb01855b4644905ce5bb713c7b60" candidate_uuid: "f58d8846a30845778b5b2ac559012ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72712f07e7c842db975ebd52c521c8b8"
I20260929 06:30:51.227334 25058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8213eb01855b4644905ce5bb713c7b60" candidate_uuid: "f58d8846a30845778b5b2ac559012ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f551c7530940d5a625b0533d9b1af1"
I20260929 06:30:51.227382 24919 raft_consensus.cc:3037] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:51.227392 25058 raft_consensus.cc:3037] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:30:51.228262 24919 raft_consensus.cc:2454] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58d8846a30845778b5b2ac559012ac7 in term 1.
I20260929 06:30:51.228262 25058 raft_consensus.cc:2454] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58d8846a30845778b5b2ac559012ac7 in term 1.
I20260929 06:30:51.228440 24721 leader_election.cc:302] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72712f07e7c842db975ebd52c521c8b8, f58d8846a30845778b5b2ac559012ac7; no voters:
I20260929 06:30:51.228523 25130 raft_consensus.cc:2781] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:30:51.228727 25130 raft_consensus.cc:686] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [term 1 LEADER]: Becoming Leader. State: Replica: f58d8846a30845778b5b2ac559012ac7, State: Running, Role: LEADER
I20260929 06:30:51.228819 25130 consensus_queue.cc:237] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } }
I20260929 06:30:51.229604 24620 catalog_manager.cc:5495] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f58d8846a30845778b5b2ac559012ac7 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "f58d8846a30845778b5b2ac559012ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f58d8846a30845778b5b2ac559012ac7" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 35919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260929 06:30:51.259562 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.257006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45594 (local address 127.13.15.129:35919)
0929 06:30:51.257062 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 06:30:51.257063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:30:51.257070 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:30:51.257114 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:30:51.257115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:30:51.257131 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:30:51.257203 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:30:51.257335 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.257812 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.258745 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:30:51.258954 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:30:51.259029 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:30:51.259159 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:30:51.259273 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:30:51.259369 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 06:30:51.259386 (+ 17us) server_negotiation.cc:299] Negotiation successful
0929 06:30:51.259406 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 06:30:51.261443 25058 raft_consensus.cc:1261] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 1 FOLLOWER]: Refusing update from remote peer f58d8846a30845778b5b2ac559012ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:30:51.261451 24919 raft_consensus.cc:1261] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 1 FOLLOWER]: Refusing update from remote peer f58d8846a30845778b5b2ac559012ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:30:51.261816 25130 consensus_queue.cc:1035] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72712f07e7c842db975ebd52c521c8b8" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 36629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:30:51.261984 25134 consensus_queue.cc:1035] T 8213eb01855b4644905ce5bb713c7b60 P f58d8846a30845778b5b2ac559012ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55f551c7530940d5a625b0533d9b1af1" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:30:51.265247 25137 mvcc.cc:205] Tried to move back new op lower bound from 7334557496364302336 to 7334557496233799680. Current Snapshot: MvccSnapshot[applied={T|T < 7334557496364302336}]
I20260929 06:30:51.266244 25140 mvcc.cc:205] Tried to move back new op lower bound from 7334557496364302336 to 7334557496233799680. Current Snapshot: MvccSnapshot[applied={T|T < 7334557496364302336}]
I20260929 06:30:51.268782 24619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:30:51.268893 24619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:30:51.269557 24619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0fd9390c9c84ee2b8f258e1cbdc8706: Sending DeleteTablet for 3 replicas of tablet 8213eb01855b4644905ce5bb713c7b60
I20260929 06:30:51.269872 24766 tablet_service.cc:1514] Processing DeleteTablet for tablet 8213eb01855b4644905ce5bb713c7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:45582
I20260929 06:30:51.269979 24895 tablet_service.cc:1514] Processing DeleteTablet for tablet 8213eb01855b4644905ce5bb713c7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:50090
I20260929 06:30:51.270195 25038 tablet_service.cc:1514] Processing DeleteTablet for tablet 8213eb01855b4644905ce5bb713c7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:30:51 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:39050
I20260929 06:30:51.270394 25148 tablet_replica.cc:330] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: stopping tablet replica
I20260929 06:30:51.270491 25148 raft_consensus.cc:2229] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:30:51.270586 25148 raft_consensus.cc:2258] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:30:51.270748 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 24701
I20260929 06:30:51.271003 25148 ts_tablet_manager.cc:1905] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:30:51.273023 25147 tablet_replica.cc:330] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: stopping tablet replica
I20260929 06:30:51.273204 25147 raft_consensus.cc:2229] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:30:51.273386 25147 raft_consensus.cc:2258] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:30:51.273468 25148 ts_tablet_manager.cc:1918] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:30:51.273571 25148 log.cc:1196] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/8213eb01855b4644905ce5bb713c7b60
I20260929 06:30:51.274380 25147 ts_tablet_manager.cc:1905] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:30:51.274408 25148 ts_tablet_manager.cc:1939] T 8213eb01855b4644905ce5bb713c7b60 P 55f551c7530940d5a625b0533d9b1af1: Deleting consensus metadata
I20260929 06:30:51.274783 24606 catalog_manager.cc:4841] TS 55f551c7530940d5a625b0533d9b1af1 (127.13.15.131:33395): tablet 8213eb01855b4644905ce5bb713c7b60 (table test-table [id=f119792567974286814326e5af2b38db]) successfully deleted
I20260929 06:30:51.276063 25147 ts_tablet_manager.cc:1918] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:30:51.276136 25147 log.cc:1196] T 8213eb01855b4644905ce5bb713c7b60 P 72712f07e7c842db975ebd52c521c8b8: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/8213eb01855b4644905ce5bb713c7b60
W20260929 06:30:51.277318 24607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:35919 (error 108)
I20260929 06:30:51.277536 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 24840
I20260929 06:30:51.278424 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.278310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:35919 (local address 127.0.0.1:45604)
0929 06:30:51.278353 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.278383 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:35919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 06:30:51.278541 24607 catalog_manager.cc:4584] TS f58d8846a30845778b5b2ac559012ac7 (127.13.15.129:35919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8213eb01855b4644905ce5bb713c7b60: Network error: Client connection negotiation failed: client connection to 127.13.15.129:35919: connect: Connection refused (error 111)
I20260929 06:30:51.283613 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:30:51.283516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:36629 (local address 127.0.0.1:50104)
0929 06:30:51.283559 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:30:51.283583 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.130:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 06:30:51.283708 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 24975
I20260929 06:30:51.289582 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 24588
I20260929 06:30:51.295063 13374 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 06:30:51.629056 13374 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 06:30:51.629186 13374 mini_ranger.cc:66] Stopping Ranger...
I20260929 06:30:51.974303 13374 mini_ranger.cc:68] Stopped Ranger
2026-09-29 06:30:51.974 UTC [24174] LOG: received smart shutdown request
2026-09-29 06:30:51.978 UTC [24174] LOG: background worker "logical replication launcher" (PID 24183) exited with exit code 1
2026-09-29 06:30:51.978 UTC [24178] LOG: shutting down
2026-09-29 06:30:52.140 UTC [24174] LOG: database system is shut down
2026-09-29T06:30:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46299 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:30:52 dist-test-slave-sm0p krb5kdc[25158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:30:52 dist-test-slave-sm0p krb5kdc[25158](info): set up 2 sockets
Sep 29 06:30:52 dist-test-slave-sm0p krb5kdc[25158](info): commencing operation
krb5kdc: starting...
W20260929 06:30:53.289932 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:30:53 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:30:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:30:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangeradmin_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangeradmin_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangerlookup_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangerlookup_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
I20260929 06:30:53.448242 13374 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJYAK93/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/postgres -l logfile start
2026-09-29 06:30:55.885 UTC [25188] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 06:30:55.885 UTC [25188] LOG: listening on IPv4 address "127.13.15.148", port 45077
2026-09-29 06:30:55.889 UTC [25188] LOG: listening on Unix socket "/tmp/.s.PGSQL.45077"
2026-09-29 06:30:55.923 UTC [25191] LOG: database system was shut down at 2026-09-29 06:30:54 UTC
2026-09-29 06:30:55.927 UTC [25188] LOG: database system is ready to accept connections
I20260929 06:30:56.861969 13374 mini_postgres.cc:96] Postgres bound to 45077
2026-09-29 06:30:56.868 UTC [25199] FATAL: database "slave" does not exist
127.13.15.148:45077 - accepting connections
I20260929 06:30:56.868507 13374 mini_ranger.cc:161] Starting Ranger...
I20260929 06:30:56.882593 13374 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 06:30:57.342726 13374 mini_ranger.cc:87] Created ranger Postgres database
I20260929 06:30:57.342821 13374 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin
2026-09-29 06:30:57,596 [I] DB FLAVOR :POSTGRES
2026-09-29 06:30:57,596 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 06:30:57,597 [I] Checking connection..
2026-09-29 06:30:57,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 06:30:57,826 [I] Checking connection passed.
2026-09-29 06:30:57,826 [I] --------- Verifying version history table ---------
2026-09-29 06:30:57,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:58,067 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 06:30:58,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:58,319 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 06:30:58,319 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 06:30:58,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 06:30:58.583 UTC [25263] WARNING: there is no transaction in progress
2026-09-29 06:30:58,589 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 06:30:58,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 06:30:58,828 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 06:30:58,828 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 06:30:58,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 06:30:59,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 06:30:59,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sm0p.c.gcp-upstream.internal','N') ;"
2026-09-29 06:30:59,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 06:30:59,761 [I] Table x_portal_user does not exist in database ranger
2026-09-29 06:30:59,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 06:31:00,007 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 06:31:00,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 06:31:00,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 06:31:00,513 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 06:31:00,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 06:31:01.456 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.473 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.490 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.507 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.524 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.542 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.597 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.603 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.617 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.627 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.637 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.652 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.661 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.667 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.674 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:01.681 UTC [25371] WARNING: there is no transaction in progress
2026-09-29 06:31:02,308 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 06:31:02,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 06:31:02,555 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 06:31:02,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 06:31:02,811 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 06:31:02,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 06:31:03,055 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 06:31:03,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 06:31:03,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 06:31:03,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sm0p.c.gcp-upstream.internal';"
2026-09-29 06:31:03,793 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 06:31:03,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 06:31:04,032 [I] DB_PATCHES have already been applied
I20260929 06:31:04.037452 13374 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJYAK93/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 06:31:04.037509 13374 mini_ranger.cc:193] Using host: 127.13.15.148
I20260929 06:31:04.040442 13374 mini_ranger.cc:227] Ranger admin URL: http://127.13.15.148:56517
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 6:31:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 06:31:05.147538 13374 mini_ranger.cc:160] Time spent starting Ranger: real 8.279s user 0.000s sys 0.008s
Sep 29, 2026 6:31:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 6:31:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 29, 2026 6:31:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 6:31:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 6:31:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 06:31:26.968467 13374 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangerkms_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/rangerkms_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.15.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.15.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.15.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/HTTP_127.13.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 06:31:27 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663487, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 06:31:27.051021 13374 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 06:31:27.059650 13374 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 06:31:27.917696 13374 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 06:31:27.917801 13374 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms
I20260929 06:31:27.917824 13374 mini_ranger_kms.cc:227] Using postgres at 127.13.15.148:45077
2026-09-29 06:31:28,080 [I] DB FLAVOR :POSTGRES
2026-09-29 06:31:28,080 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 06:31:28,080 [I] Checking connection
2026-09-29 06:31:28,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 06:31:28,308 [I] connection success
2026-09-29 06:31:28,308 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 06:31:28,308 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 06:31:28,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 06:31:28,537 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 06:31:28,537 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 06:31:28,537 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 06:31:28,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.15.148:45077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 06:31:28,787 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 06:31:29.232290 13374 mini_ranger_kms.cc:310] Created kms service
I20260929 06:31:29.397914 13374 mini_ranger_kms.cc:326] Created kudu user
I20260929 06:31:29.439296 13374 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 06:31:29 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663487, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:29.838284 13374 mini_ranger_kms.cc:384] Added ranger policy
I20260929 06:31:29.838505 13374 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJYAK93/build/release/bin/postgresql.jar:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJYAK93/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 06:31:29.838528 13374 mini_ranger_kms.cc:240] Using host: 127.13.15.148
I20260929 06:31:29.841928 13374 mini_ranger_kms.cc:276] Ranger KMS PID: 25551
I20260929 06:31:29.842005 13374 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.15.148:34859
Sep 29, 2026 6:31:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 6:31:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 6:31:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 6:31:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 6:31:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 6:31:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 06:31:32.157709 13374 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.107s user 0.007s sys 0.139s
I20260929 06:31:32.157845 13374 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 06:31:32.157898 13374 mini_ranger_kms.cc:401] 127.13.15.148:34859/kms/v1/keys
Sep 29 06:31:32 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790663492, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 06:31:32 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663492, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
Sep 29 06:31:33 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663492, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.15.148@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
Sep 29 06:31:35 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:31:35.244800 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44107
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44107
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:35.323333 25602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:35.323582 25602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:35.323616 25602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:35.324764 25602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:35.324831 25602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:35.324852 25602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:35.324872 25602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:35.324898 25602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:35.326454 25602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.15.148:34859/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44107
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:35.326710 25602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:35.326965 25602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:35.332147 25608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:35.332222 25602 server_base.cc:1034] running on GCE node
W20260929 06:31:35.332408 25610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:35.332156 25607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:35.332758 25602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:35.333041 25602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:35.334187 25602 hybrid_clock.cc:648] HybridClock initialized: now 1790663495334172 us; error 30 us; skew 500 ppm
Sep 29 06:31:35 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663495, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:35.336647 25602 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:35.337126 25602 webserver.cc:460] Webserver started at http://127.13.15.190:41997/ using document root <none> and password file <none>
I20260929 06:31:35.337357 25602 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:35.337402 25602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:35.337529 25602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:31:35 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663495, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:35.394018 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "c623b195661f4fb89e8bfd7035d1a75a"
format_stamp: "Formatted at 2026-09-29 06:31:35 on dist-test-slave-sm0p"
server_key: "c7907611834b805f7fca92336be44343"
server_key_iv: "339650061f7ed8728863da4a50d14ff4"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:35.394515 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "c623b195661f4fb89e8bfd7035d1a75a"
format_stamp: "Formatted at 2026-09-29 06:31:35 on dist-test-slave-sm0p"
server_key: "c7907611834b805f7fca92336be44343"
server_key_iv: "339650061f7ed8728863da4a50d14ff4"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:35.395955 25602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:35.407088 25617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:35.407317 25602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:31:35.407439 25602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "c623b195661f4fb89e8bfd7035d1a75a"
format_stamp: "Formatted at 2026-09-29 06:31:35 on dist-test-slave-sm0p"
server_key: "c7907611834b805f7fca92336be44343"
server_key_iv: "339650061f7ed8728863da4a50d14ff4"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:35.407532 25602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:35.428211 25602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:35.430084 25602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:35.430294 25602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:35.435081 25602 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44107
I20260929 06:31:35.435174 25669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44107 every 8 connection(s)
I20260929 06:31:35.435514 25602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:35.436205 25670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:35.438490 25670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Bootstrap starting.
I20260929 06:31:35.439065 25670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:35.439343 25670 log.cc:824] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:35.440028 25670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: No bootstrap required, opened a new log
I20260929 06:31:35.440253 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 25602
I20260929 06:31:35.440364 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:31:35.440558 13374 external_mini_cluster.cc:1428] Setting key edba5c3ba961aa7555e0b81941ce6969
I20260929 06:31:35.442477 25670 raft_consensus.cc:348] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:35.442627 25670 raft_consensus.cc:374] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:35.442690 25670 raft_consensus.cc:729] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c623b195661f4fb89e8bfd7035d1a75a, State: Initialized, Role: FOLLOWER
I20260929 06:31:35.442865 25670 consensus_queue.cc:260] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:35.442961 25670 raft_consensus.cc:388] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:35.443014 25670 raft_consensus.cc:482] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:35.443096 25670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:35.443663 25670 raft_consensus.cc:504] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:35.443768 25670 leader_election.cc:302] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c623b195661f4fb89e8bfd7035d1a75a; no voters:
I20260929 06:31:35.443955 25670 leader_election.cc:288] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:31:35.444013 25675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:35.444198 25675 raft_consensus.cc:686] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 LEADER]: Becoming Leader. State: Replica: c623b195661f4fb89e8bfd7035d1a75a, State: Running, Role: LEADER
I20260929 06:31:35.444248 25670 sys_catalog.cc:564] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:35.444373 25675 consensus_queue.cc:237] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:35.444746 25675 sys_catalog.cc:455] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c623b195661f4fb89e8bfd7035d1a75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } } }
I20260929 06:31:35.444878 25675 sys_catalog.cc:458] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:35.445041 25675 sys_catalog.cc:455] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c623b195661f4fb89e8bfd7035d1a75a. Latest consensus state: current_term: 1 leader_uuid: "c623b195661f4fb89e8bfd7035d1a75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } } }
I20260929 06:31:35.445144 25675 sys_catalog.cc:458] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:35.445289 25684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:35.445637 25670 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 06:31:35.445875 25684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:31:35 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:35.454579 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:35.441053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.13.15.190:44107)
0929 06:31:35.441237 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:35.441246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:35.441277 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:31:35.442491 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:35.442501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:35.442569 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:35.442720 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:35.443586 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:35.444998 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:35.446216 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:35.446651 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:35.447067 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:35.449821 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:35.449855 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:35.449855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:35.449924 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:35.452362 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:35.452694 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:35.452697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:35.452697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:35.452791 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:35.452996 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:35.452997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:35.452997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:35.453294 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:35.453441 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:35.453541 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 06:31:35.453999 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260929 06:31:35.459447 25692 mvcc.cc:205] Tried to move back new op lower bound from 7334557677350346752 to 7334557677340721152. Current Snapshot: MvccSnapshot[applied={T|T < 7334557677350346752}]
I20260929 06:31:35.460251 25684 catalog_manager.cc:1347] Generated new cluster ID: f4a39f65f9b240489d2cc4fe697ba47b
I20260929 06:31:35.460297 25684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:31:35.472723 25684 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:31:35.473210 25684 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:35.481204 25684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Generated new TSK 0
I20260929 06:31:35.481458 25684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:31:36.214089 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:36.333639 25715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:36.333909 25715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:36.333940 25715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:36.335616 25715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:36.335675 25715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:36.335713 25715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:31:36.337761 25715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:36.338284 25715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:36.338621 25715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:36.345104 25723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.345083 25720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.345222 25721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:36.345713 25715 server_base.cc:1034] running on GCE node
I20260929 06:31:36.345903 25715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:36.346180 25715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:36.347357 25715 hybrid_clock.cc:648] HybridClock initialized: now 1790663496347329 us; error 41 us; skew 500 ppm
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:36.350384 25715 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:31:36.350780 25715 webserver.cc:460] Webserver started at http://127.13.15.129:41197/ using document root <none> and password file <none>
I20260929 06:31:36.351015 25715 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:36.351078 25715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:36.351214 25715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:36.363185 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "1f7d921da6534fd7bc9140df6f00d15e"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "33a2e38ec00e9a8b0fe4ebb8993a83f8"
server_key_iv: "d6c679c32f92eec09f1633fd31eb702a"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.363669 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "1f7d921da6534fd7bc9140df6f00d15e"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "33a2e38ec00e9a8b0fe4ebb8993a83f8"
server_key_iv: "d6c679c32f92eec09f1633fd31eb702a"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.368409 25715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260929 06:31:36.381929 25730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.382109 25715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 06:31:36.382238 25715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "1f7d921da6534fd7bc9140df6f00d15e"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "33a2e38ec00e9a8b0fe4ebb8993a83f8"
server_key_iv: "d6c679c32f92eec09f1633fd31eb702a"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.382365 25715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:36.396947 25715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:36.399062 25715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:36.399224 25715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:36.399468 25715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:36.399847 25715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:36.399901 25715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.399977 25715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:36.400038 25715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.407425 25715 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38603
I20260929 06:31:36.407967 25715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:31:36.409271 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38603 every 8 connection(s)
I20260929 06:31:36.417006 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 25715
I20260929 06:31:36.417104 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:31:36.417312 13374 external_mini_cluster.cc:1428] Setting key 1988c9a4ea24b0a125cec192b310a9d2
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:36.423576 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.409936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:59579 (local address 127.13.15.190:44107)
0929 06:31:36.410088 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:36.410091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:36.410298 (+ 207us) server_negotiation.cc:407] Connection header received
0929 06:31:36.411036 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:36.411038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:36.411053 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:36.411097 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:36.412732 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.413570 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.415092 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.415437 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.415487 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.418096 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:36.418097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:36.418098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:36.418105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:36.420954 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.422321 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.422323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.422323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.422390 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.422723 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.422724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.422724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.422887 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:36.422944 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:36.423333 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 06:31:36.423432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260929 06:31:36.424567 25847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.409930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.129:59579)
0929 06:31:36.410170 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:36.410215 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:36.410902 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:36.411311 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:36.411316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:36.411566 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:36.411845 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.411852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.413688 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.413688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.414932 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.414935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.415595 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.415596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.415736 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.416310 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:36.416320 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:36.418001 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:36.421863 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.421871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.421874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.422221 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.422499 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.422499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.422500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.422639 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.422982 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:36.422988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:36.423165 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 06:31:36.424086 (+ 921us) client_negotiation.cc:240] Negotiation successful
0929 06:31:36.424265 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260929 06:31:36.425308 25844 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:36.425464 25844 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:36.425707 25844 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:36.426326 25634 ts_manager.cc:194] Registered new tserver with Master: 1f7d921da6534fd7bc9140df6f00d15e (127.13.15.129:38603)
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:31:36.427130 25634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:59579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:31:36.445284 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:36.557026 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:36.557303 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:36.557337 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:36.558966 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:36.559017 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:36.559053 25852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:31:36.561125 25852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:36.561371 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:36.561671 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:36.567766 25859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.567808 25860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.568073 25862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:36.568737 25852 server_base.cc:1034] running on GCE node
I20260929 06:31:36.568974 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:36.569254 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:36.570426 25852 hybrid_clock.cc:648] HybridClock initialized: now 1790663496570404 us; error 35 us; skew 500 ppm
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:36.573814 25852 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:31:36.574436 25852 webserver.cc:460] Webserver started at http://127.13.15.130:35661/ using document root <none> and password file <none>
I20260929 06:31:36.574756 25852 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:36.574859 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:36.575037 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:36.594877 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "14517eaf421746f9b1844283f25d28ee"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "1d275c4d5e731cf1c605bb1f81556649"
server_key_iv: "7c540dc5ff2529dd953f5e71821b5435"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.595687 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "14517eaf421746f9b1844283f25d28ee"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "1d275c4d5e731cf1c605bb1f81556649"
server_key_iv: "7c540dc5ff2529dd953f5e71821b5435"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.597724 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:36.618507 25869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.618765 25852 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260929 06:31:36.618855 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "14517eaf421746f9b1844283f25d28ee"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "1d275c4d5e731cf1c605bb1f81556649"
server_key_iv: "7c540dc5ff2529dd953f5e71821b5435"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.618943 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:36.637943 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:36.640383 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:36.640563 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:36.640816 25852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:36.641204 25852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:36.641294 25852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.641371 25852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:36.641422 25852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.648077 25852 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:34923
I20260929 06:31:36.648546 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:31:36.650483 25982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:34923 every 8 connection(s)
I20260929 06:31:36.652941 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 25852
I20260929 06:31:36.653036 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:31:36.653224 13374 external_mini_cluster.cc:1428] Setting key 370d7667745936dbec2f9135ab7f4c63
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:36.660832 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.651201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:53487 (local address 127.13.15.190:44107)
0929 06:31:36.651387 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:36.651390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:36.651433 (+ 43us) server_negotiation.cc:407] Connection header received
0929 06:31:36.652085 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:36.652087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:36.652103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:36.652145 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:36.652940 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.653721 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.655183 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.655490 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.655533 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.658102 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:36.658104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:36.658104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:36.658112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:36.659494 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.659890 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.659891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.659891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.659951 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.660183 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.660184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.660185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.660296 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:36.660370 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:36.660640 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 06:31:36.660743 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260929 06:31:36.661051 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.651171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.130:53487)
0929 06:31:36.651312 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:36.651351 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:36.651966 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:36.652327 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:36.652331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:36.652549 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:36.652825 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.652831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.653834 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.653835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.655084 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.655086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.655631 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.655632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.655765 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.656338 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:36.656348 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:36.657995 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:36.659594 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.659600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.659602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.659809 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.660014 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.660015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.660015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.660119 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.660362 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:36.660367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:36.660495 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 06:31:36.660621 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 06:31:36.660762 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260929 06:31:36.661554 25983 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:36.661648 25983 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:36.661827 25983 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:36.662221 25634 ts_manager.cc:194] Registered new tserver with Master: 14517eaf421746f9b1844283f25d28ee (127.13.15.130:34923)
I20260929 06:31:36.662680 25634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:53487
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:31:36.687076 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:36.806802 25989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:36.807190 25989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:36.807294 25989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:36.809012 25989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:36.809128 25989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:36.809228 25989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:31:36.811520 25989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:36.812086 25989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:36.812400 25989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:36.818698 25995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.818804 25994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:36.818706 25997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:36.819829 25989 server_base.cc:1034] running on GCE node
I20260929 06:31:36.820067 25989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:36.820384 25989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:36.821565 25989 hybrid_clock.cc:648] HybridClock initialized: now 1790663496821537 us; error 41 us; skew 500 ppm
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:36.824887 25989 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:31:36.825465 25989 webserver.cc:460] Webserver started at http://127.13.15.131:44935/ using document root <none> and password file <none>
I20260929 06:31:36.825755 25989 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:36.825819 25989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:36.825938 25989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:36.838793 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "f522f682d61a447197e588e5ecfcf5c9"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "6985d8b3c40a3ae32712101bbad2d241"
server_key_iv: "2ae53af97677ba33d975d63ba76079b2"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.839326 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "f522f682d61a447197e588e5ecfcf5c9"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "6985d8b3c40a3ae32712101bbad2d241"
server_key_iv: "2ae53af97677ba33d975d63ba76079b2"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.842586 25989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 06:31:36.861167 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.861423 25989 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260929 06:31:36.861514 25989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "f522f682d61a447197e588e5ecfcf5c9"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "6985d8b3c40a3ae32712101bbad2d241"
server_key_iv: "2ae53af97677ba33d975d63ba76079b2"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:36.861603 25989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:36.879546 25989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:36.881536 25989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:36.881680 25989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:36.881928 25989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:36.882419 25989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:36.882457 25989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.882479 25989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:36.882491 25989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:36.889741 25989 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:36931
I20260929 06:31:36.889827 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:36931 every 8 connection(s)
I20260929 06:31:36.890126 25989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:31:36.894692 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 25989
I20260929 06:31:36.894795 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:31:36.895012 13374 external_mini_cluster.cc:1428] Setting key 43aff299ee2010c90d383a3190f8f86b
Sep 29 06:31:36 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:36.899349 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.890893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:59623 (local address 127.13.15.190:44107)
0929 06:31:36.890961 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:36.890964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:36.891143 (+ 179us) server_negotiation.cc:407] Connection header received
0929 06:31:36.891653 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:36.891654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:36.891667 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:36.891735 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:36.892289 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.892770 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.893782 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.893993 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.894064 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.896476 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:36.896478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:36.896478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:36.896485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:36.897906 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.898377 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.898378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.898378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.898442 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:36.898717 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:36.898718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:36.898719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:36.898835 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:36.898909 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:36.899182 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 06:31:36.899264 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:31:36.914696 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:36.890871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.131:59623)
0929 06:31:36.891037 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:36.891075 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:36.891546 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:36.891806 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:36.891810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:36.891994 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:36.892185 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.892190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.892860 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.892861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.893672 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:36.893675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:36.894112 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:36.894113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:36.894289 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:36.894894 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:36.894907 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:36.896368 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:36.897992 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.897998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.898001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.898302 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.898505 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:36.898505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:36.898506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:36.898643 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:36.898900 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:36.898906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:36.899049 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 06:31:36.899161 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 06:31:36.914441 (+ 15280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260929 06:31:36.915270 26118 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:36.915398 26118 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:36.915603 26118 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:36.916018 25634 ts_manager.cc:194] Registered new tserver with Master: f522f682d61a447197e588e5ecfcf5c9 (127.13.15.131:36931)
I20260929 06:31:36.916515 25634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:59623
I20260929 06:31:36.918252 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:31:36.919052 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 25715
I20260929 06:31:36.925413 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 25852
I20260929 06:31:36.931010 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 25989
I20260929 06:31:36.938032 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 25602
I20260929 06:31:36.942636 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44107
--webserver_interface=127.13.15.190
--webserver_port=41997
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44107
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:37.021781 26121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:37.021968 26121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:37.021989 26121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:37.023149 26121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:37.023190 26121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:37.023203 26121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:37.023216 26121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:37.023226 26121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:37.024773 26121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44107
--ranger_config_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=41997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:37.025007 26121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:37.025219 26121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:37.029961 26126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.030004 26127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:37.030009 26121 server_base.cc:1034] running on GCE node
W20260929 06:31:37.029980 26129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:37.030440 26121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:37.030743 26121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:37.031891 26121 hybrid_clock.cc:648] HybridClock initialized: now 1790663497031874 us; error 29 us; skew 500 ppm
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:37.034268 26121 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:37.034699 26121 webserver.cc:460] Webserver started at http://127.13.15.190:41997/ using document root <none> and password file <none>
I20260929 06:31:37.034919 26121 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:37.034991 26121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:37.074678 26121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:37.075524 26137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.075729 26121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:37.075836 26121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "c623b195661f4fb89e8bfd7035d1a75a"
format_stamp: "Formatted at 2026-09-29 06:31:35 on dist-test-slave-sm0p"
server_key: "c7907611834b805f7fca92336be44343"
server_key_iv: "339650061f7ed8728863da4a50d14ff4"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:37.076292 26121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:37.114111 26121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:37.115916 26121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:37.116026 26121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:37.120235 26121 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44107
I20260929 06:31:37.120276 26189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44107 every 8 connection(s)
I20260929 06:31:37.120602 26121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:37.121749 26190 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:31:37.122465 26190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Bootstrap starting.
I20260929 06:31:37.128245 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26121
I20260929 06:31:37.128520 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:38603
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:44107
--webserver_port=41197
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:37.130111 26190 log.cc:824] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:37.131626 26190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:31:37.131899 26190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Bootstrap complete.
I20260929 06:31:37.134469 26190 raft_consensus.cc:348] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:37.134709 26190 raft_consensus.cc:729] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c623b195661f4fb89e8bfd7035d1a75a, State: Initialized, Role: FOLLOWER
I20260929 06:31:37.134910 26190 consensus_queue.cc:260] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:37.135018 26190 raft_consensus.cc:388] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:37.135069 26190 raft_consensus.cc:482] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:37.135114 26190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:31:37.136016 26190 raft_consensus.cc:504] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:37.136125 26190 leader_election.cc:302] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c623b195661f4fb89e8bfd7035d1a75a; no voters:
I20260929 06:31:37.136296 26190 leader_election.cc:288] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 06:31:37.136343 26194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 2 FOLLOWER]: Leader election won for term 2
I20260929 06:31:37.136529 26194 raft_consensus.cc:686] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [term 2 LEADER]: Becoming Leader. State: Replica: c623b195661f4fb89e8bfd7035d1a75a, State: Running, Role: LEADER
I20260929 06:31:37.136528 26190 sys_catalog.cc:564] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:37.136744 26194 consensus_queue.cc:237] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } }
I20260929 06:31:37.137151 26195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c623b195661f4fb89e8bfd7035d1a75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } } }
I20260929 06:31:37.137310 26196 sys_catalog.cc:455] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c623b195661f4fb89e8bfd7035d1a75a. Latest consensus state: current_term: 2 leader_uuid: "c623b195661f4fb89e8bfd7035d1a75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c623b195661f4fb89e8bfd7035d1a75a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44107 } } }
I20260929 06:31:37.137589 26195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:37.137792 26196 sys_catalog.cc:458] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:37.137802 26204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:37.138388 26204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:31:37.138506 26204 catalog_manager.cc:1259] Loaded cluster ID: f4a39f65f9b240489d2cc4fe697ba47b
I20260929 06:31:37.138530 26204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:31:37.138697 26190 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 06:31:37.142612 26204 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:37.142874 26204 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Loaded TSK: 0
I20260929 06:31:37.143096 26204 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:31:37.216756 26192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:37.217041 26192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:37.217079 26192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:37.218277 26192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:37.218331 26192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:37.218372 26192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:31:37.220178 26192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:38603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=41197
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:37.220427 26192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:37.220639 26192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:37.225900 26230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.225922 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.226248 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:37.226828 26192 server_base.cc:1034] running on GCE node
I20260929 06:31:37.227023 26192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:37.227267 26192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:37.228435 26192 hybrid_clock.cc:648] HybridClock initialized: now 1790663497228417 us; error 32 us; skew 500 ppm
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:37.234987 26192 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:31:37.235461 26192 webserver.cc:460] Webserver started at http://127.13.15.129:41197/ using document root <none> and password file <none>
I20260929 06:31:37.235757 26192 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:37.235893 26192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:37.251854 26192 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:37.252835 26238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.253019 26192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:31:37.253105 26192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "1f7d921da6534fd7bc9140df6f00d15e"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "33a2e38ec00e9a8b0fe4ebb8993a83f8"
server_key_iv: "d6c679c32f92eec09f1633fd31eb702a"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:37.253530 26192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:37.272481 26192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:37.274756 26192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:37.274981 26192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:37.275288 26192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:37.275681 26192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:37.275766 26192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.275875 26192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:37.275947 26192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.283217 26192 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38603
I20260929 06:31:37.283280 26351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38603 every 8 connection(s)
I20260929 06:31:37.283635 26192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:31:37.286804 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26192
I20260929 06:31:37.287112 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:34923
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:44107
--webserver_port=35661
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:37.299654 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.284554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:45739 (local address 127.13.15.190:44107)
0929 06:31:37.284749 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:37.284760 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:37.284955 (+ 195us) server_negotiation.cc:407] Connection header received
0929 06:31:37.285595 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:37.285600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:37.285643 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:37.285774 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:37.286912 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.290990 (+ 4078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.292381 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.292961 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.293056 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.295455 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:37.295460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:37.295460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:37.295472 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:37.297531 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.298051 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.298054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.298055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.298180 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.298478 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.298479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.298480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.298779 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:37.298850 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:37.299283 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 06:31:37.299417 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260929 06:31:37.300511 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.284500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.129:45739)
0929 06:31:37.284823 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:37.284852 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:37.285333 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:37.285812 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:37.285817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:37.286099 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:37.286413 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.286420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.291096 (+ 4676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.291098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.292287 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.292289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.293188 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.293189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.293317 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.293888 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:37.293898 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:37.295317 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:37.297633 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.297640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.297642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.297955 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.298265 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.298266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.298266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.298396 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.298900 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:37.298906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:37.299042 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 06:31:37.300135 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0929 06:31:37.300292 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":75,"threads_started":1}
I20260929 06:31:37.301470 26352 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:37.301553 26352 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.301743 26352 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.302701 26352 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44107 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 06:31:37.302770 26352 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.302830 26352 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:37.303261 26352 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.303334 26352 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.303714 26352 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:31:37.415043 26356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:37.415321 26356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:37.415367 26356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:37.417088 26356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:37.417155 26356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:37.417202 26356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:31:37.419324 26356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:34923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=35661
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:37.419570 26356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:37.419934 26356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:37.426292 26364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.426522 26362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.426362 26361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:37.426872 26356 server_base.cc:1034] running on GCE node
I20260929 06:31:37.427122 26356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:37.427421 26356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:37.428689 26356 hybrid_clock.cc:648] HybridClock initialized: now 1790663497428667 us; error 34 us; skew 500 ppm
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:37.431881 26356 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:31:37.432461 26356 webserver.cc:460] Webserver started at http://127.13.15.130:35661/ using document root <none> and password file <none>
I20260929 06:31:37.432760 26356 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:37.432843 26356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:37.448354 26356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:37.462199 26372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.462483 26356 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260929 06:31:37.462584 26356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "14517eaf421746f9b1844283f25d28ee"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "1d275c4d5e731cf1c605bb1f81556649"
server_key_iv: "7c540dc5ff2529dd953f5e71821b5435"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:37.463066 26356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:37.476127 26356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:37.481686 26356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:37.481921 26356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:37.482556 26356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:37.483000 26356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:37.483111 26356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.483196 26356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:37.483268 26356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.491041 26356 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:34923
I20260929 06:31:37.491510 26356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:31:37.493072 26485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:34923 every 8 connection(s)
I20260929 06:31:37.498260 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26356
I20260929 06:31:37.498555 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.15.148:34859/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:36931
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:44107
--webserver_port=44935
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:37.507637 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.493018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:54099 (local address 127.13.15.190:44107)
0929 06:31:37.493068 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:37.493070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:37.493857 (+ 787us) server_negotiation.cc:407] Connection header received
0929 06:31:37.494606 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:37.494607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:37.494625 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:37.494717 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:37.495473 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.496236 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.497728 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.498062 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.498111 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.504125 (+ 6014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:37.504127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:37.504127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:37.504137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:37.505943 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.506517 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.506518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.506519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.506585 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.506885 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.506885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.506886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.507017 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:37.507068 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:37.507455 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 06:31:37.507545 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:31:37.508190 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.492659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.130:54099)
0929 06:31:37.493718 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:37.493764 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:37.494477 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:37.494794 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:37.494798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:37.495029 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:37.495354 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.495361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.496357 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.496358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.497628 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.497630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.498315 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.498316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.498446 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.502324 (+ 3878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:37.502335 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:37.503985 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:37.506051 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.506060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.506062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.506418 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.506672 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.506673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.506674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.506810 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.507124 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:37.507129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:37.507299 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 06:31:37.507797 (+ 498us) client_negotiation.cc:240] Negotiation successful
0929 06:31:37.507961 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"thread_start_us":108,"threads_started":1}
I20260929 06:31:37.508641 26486 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:37.508741 26486 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.508941 26486 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.509517 26486 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44107 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 06:31:37.509593 26486 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.509661 26486 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:37.509948 26486 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.510025 26486 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.510397 26486 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:31:37.624328 26489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:37.624723 26489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:37.624809 26489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:37.626542 26489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:37.626653 26489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:37.626740 26489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:31:37.628999 26489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.15.148:34859/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:36931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=44935
--tserver_master_addrs=127.13.15.190:44107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:37.629479 26489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:37.629829 26489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:37.638460 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.638724 26494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:37.641163 26497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:37.647457 26489 server_base.cc:1034] running on GCE node
I20260929 06:31:37.647823 26489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:37.648124 26489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:37.663599 26489 hybrid_clock.cc:648] HybridClock initialized: now 1790663497663557 us; error 55 us; skew 500 ppm
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:37.667142 26489 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:31:37.667770 26489 webserver.cc:460] Webserver started at http://127.13.15.131:44935/ using document root <none> and password file <none>
I20260929 06:31:37.668040 26489 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:37.668121 26489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:37.699213 26489 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260929 06:31:37.700400 26506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.700641 26489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 06:31:37.700737 26489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "f522f682d61a447197e588e5ecfcf5c9"
format_stamp: "Formatted at 2026-09-29 06:31:36 on dist-test-slave-sm0p"
server_key: "6985d8b3c40a3ae32712101bbad2d241"
server_key_iv: "2ae53af97677ba33d975d63ba76079b2"
server_key_version: "kuduclusterkey@0"
I20260929 06:31:37.701231 26489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:37.742626 26489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:37.745088 26489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:37.745257 26489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:37.745530 26489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:37.745916 26489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:37.745965 26489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.745996 26489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:37.746073 26489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:37.752801 26489 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:36931
I20260929 06:31:37.753332 26489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:31:37.757475 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26489
I20260929 06:31:37.754882 26619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:36931 every 8 connection(s)
Sep 29 06:31:37 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:37.771190 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.755776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:39501 (local address 127.13.15.190:44107)
0929 06:31:37.755824 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:37.755826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:37.760353 (+ 4527us) server_negotiation.cc:407] Connection header received
0929 06:31:37.761028 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:37.761029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:37.761043 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:37.761127 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:37.761941 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.762718 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.764257 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.764573 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.764624 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.767318 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:37.767319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:37.767320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:37.767327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:37.769393 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.769885 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.769886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.769887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.769941 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.770287 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.770288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.770288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.770416 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:37.770461 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:37.771016 (+ 555us) server_negotiation.cc:299] Negotiation successful
0929 06:31:37.771097 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:31:37.771965 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.755468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44107 (local address 127.13.15.131:39501)
0929 06:31:37.760196 (+ 4728us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:37.760257 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:37.760900 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:37.761296 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:37.761301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:37.761531 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:37.761822 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.761828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.762837 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.762838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.764153 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:37.764157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.764737 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.764738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:37.764891 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.765492 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:37.765503 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:37.767206 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:37.769486 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.769492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.769494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.769781 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.770015 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:37.770015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:37.770015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:37.770186 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:37.770512 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:37.770518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:37.770833 (+ 315us) client_negotiation.cc:769] Sending connection context
0929 06:31:37.771543 (+ 710us) client_negotiation.cc:240] Negotiation successful
0929 06:31:37.771740 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4679,"thread_start_us":92,"threads_started":1}
I20260929 06:31:37.775388 26620 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44107
I20260929 06:31:37.775482 26620 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.775686 26620 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.776361 26620 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44107 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 06:31:37.776434 26620 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.776499 26620 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:37.776865 26620 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:37.776937 26620 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
W20260929 06:31:37.777262 26620 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:31:37.785384 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:37.757854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.13.15.190:44107)
0929 06:31:37.766926 (+ 9072us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:37.766929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:37.766937 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:31:37.766965 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:37.766966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:37.766979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:37.767078 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:37.778227 (+ 11149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.778950 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.779965 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:37.780301 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:37.780373 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:37.782092 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:37.782093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:37.782094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:37.782100 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:37.783976 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.784384 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.784385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.784385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.784454 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:37.784804 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:37.784804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:37.784805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:37.784936 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:37.785089 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:37.785175 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 06:31:37.785276 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9052,"thread_start_us":69,"threads_started":1}
I20260929 06:31:38.303895 26352 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:38.304032 26352 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:38.304404 26154 ts_manager.cc:194] Registered new tserver with Master: 1f7d921da6534fd7bc9140df6f00d15e (127.13.15.129:38603)
I20260929 06:31:38.305459 26154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:45739
Sep 29 06:31:38 dist-test-slave-sm0p krb5kdc[25158](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790663498, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:38.510574 26486 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:38.510736 26486 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:38.511086 26154 ts_manager.cc:194] Registered new tserver with Master: 14517eaf421746f9b1844283f25d28ee (127.13.15.130:34923)
I20260929 06:31:38.511653 26154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:54099
Sep 29 06:31:38 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663498, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
I20260929 06:31:38.777473 26620 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:38.777709 26620 heartbeater.cc:511] Master 127.13.15.190:44107 requested a full tablet report, sending...
I20260929 06:31:38.778051 26154 ts_manager.cc:194] Registered new tserver with Master: f522f682d61a447197e588e5ecfcf5c9 (127.13.15.131:36931)
I20260929 06:31:38.778471 26154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:39501
I20260929 06:31:38.783455 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:31:38.792574 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.784479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.13.15.190:44107)
0929 06:31:38.784593 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:38.784595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:38.784628 (+ 33us) server_negotiation.cc:407] Connection header received
0929 06:31:38.784689 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:38.784690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:38.784699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:38.784756 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:38.785455 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.786269 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.787305 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.787611 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.787696 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.789309 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:38.789311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:38.789311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:38.789319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:38.791201 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:38.791617 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:38.791618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:38.791619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:38.791680 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:38.791903 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:38.791905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:38.791905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:38.792034 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:38.792128 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:38.792275 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 06:31:38.792359 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260929 06:31:38.794714 26154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:31:38.796806 26154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:31:38.812459 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38603 (local address 127.0.0.1:54788)
0929 06:31:38.807694 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:38.807737 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:38.807809 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:38.808360 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:38.808361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:38.808363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:38.808429 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.808431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.809716 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.809718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.811102 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.811105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.812099 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.812100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.812215 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.812232 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 06:31:38.812283 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:31:38.812366 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":114,"threads_started":1}
I20260929 06:31:38.813086 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.13.15.129:38603)
0929 06:31:38.808197 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:38.808202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:38.808212 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:31:38.808247 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:38.808250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:38.808265 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:38.808493 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:38.808558 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.809610 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.811212 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.811996 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.812848 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.812921 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:38.812967 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 06:31:38.813007 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":57,"threads_started":1}
I20260929 06:31:38.813671 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:36931 (local address 127.0.0.1:46820)
0929 06:31:38.808044 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:38.808052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:38.808097 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:38.808301 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:38.808305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:38.808307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:38.808382 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.808384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.809969 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.809970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.812374 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.812376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.813458 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.813458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.813519 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.813524 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:31:38.813566 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 06:31:38.813605 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":89,"threads_started":1}
I20260929 06:31:38.813998 26286 tablet_service.cc:1467] Processing CreateTablet for tablet 3c82f09634824e6c9e9236cd32c044da (DEFAULT_TABLE table=test-table [id=de7e07ba277244ec9eaf12c700f04b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:38.814231 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.13.15.131:36931)
0929 06:31:38.807953 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:38.807958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:38.808136 (+ 178us) server_negotiation.cc:407] Connection header received
0929 06:31:38.808174 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:38.808176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:38.808192 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:38.808453 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:38.808522 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.809883 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.812467 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.813349 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.814004 (+ 655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.814065 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:38.814090 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 06:31:38.814133 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":61,"threads_started":1}
I20260929 06:31:38.814411 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c82f09634824e6c9e9236cd32c044da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:38.814841 26554 tablet_service.cc:1467] Processing CreateTablet for tablet 3c82f09634824e6c9e9236cd32c044da (DEFAULT_TABLE table=test-table [id=de7e07ba277244ec9eaf12c700f04b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:38.815135 26554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c82f09634824e6c9e9236cd32c044da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:38.817091 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34923 (local address 127.0.0.1:47320)
0929 06:31:38.807922 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:38.807931 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:38.807967 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:38.813622 (+ 5655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:38.813623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:38.813625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:38.813666 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.813667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.814905 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.814906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.815902 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:38.815904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.816871 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.816872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:38.816930 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.816934 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:31:38.816977 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 06:31:38.817021 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":4,"thread_start_us":109,"threads_started":1}
I20260929 06:31:38.817410 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:38.807503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.13.15.130:34923)
0929 06:31:38.813416 (+ 5913us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:38.813421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:38.813433 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:31:38.813471 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:38.813473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:38.813489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:38.813736 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:38.813826 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.814806 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.816004 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:38.816767 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:38.817201 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:38.817267 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:38.817296 (+ 29us) server_negotiation.cc:299] Negotiation successful
0929 06:31:38.817338 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5874,"thread_start_us":93,"threads_started":1}
I20260929 06:31:38.818010 26420 tablet_service.cc:1467] Processing CreateTablet for tablet 3c82f09634824e6c9e9236cd32c044da (DEFAULT_TABLE table=test-table [id=de7e07ba277244ec9eaf12c700f04b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:38.818368 26420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c82f09634824e6c9e9236cd32c044da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:38.821007 26655 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Bootstrap starting.
I20260929 06:31:38.821327 26654 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Bootstrap starting.
I20260929 06:31:38.822021 26655 tablet_bootstrap.cc:654] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:38.822021 26654 tablet_bootstrap.cc:654] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:38.822379 26654 log.cc:824] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:38.822405 26655 log.cc:824] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:38.823164 26655 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: No bootstrap required, opened a new log
I20260929 06:31:38.823164 26654 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: No bootstrap required, opened a new log
I20260929 06:31:38.823231 26655 ts_tablet_manager.cc:1397] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:38.823231 26654 ts_tablet_manager.cc:1397] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:31:38.825495 26654 raft_consensus.cc:348] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.825495 26655 raft_consensus.cc:348] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.825652 26654 raft_consensus.cc:374] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:38.825652 26655 raft_consensus.cc:374] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:38.825686 26655 raft_consensus.cc:729] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14517eaf421746f9b1844283f25d28ee, State: Initialized, Role: FOLLOWER
I20260929 06:31:38.825686 26654 raft_consensus.cc:729] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f522f682d61a447197e588e5ecfcf5c9, State: Initialized, Role: FOLLOWER
I20260929 06:31:38.825800 26655 consensus_queue.cc:260] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.825816 26654 consensus_queue.cc:260] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.826051 26655 ts_tablet_manager.cc:1428] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:31:38.826064 26654 ts_tablet_manager.cc:1428] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:31:38.826310 26620 heartbeater.cc:503] Master 127.13.15.190:44107 was elected leader, sending a full tablet report...
I20260929 06:31:38.827207 26486 heartbeater.cc:503] Master 127.13.15.190:44107 was elected leader, sending a full tablet report...
I20260929 06:31:38.827746 26653 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Bootstrap starting.
I20260929 06:31:38.828464 26653 tablet_bootstrap.cc:654] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:38.828758 26653 log.cc:824] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:38.829484 26653 tablet_bootstrap.cc:492] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: No bootstrap required, opened a new log
I20260929 06:31:38.829574 26653 ts_tablet_manager.cc:1397] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:31:38.831446 26653 raft_consensus.cc:348] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.831609 26653 raft_consensus.cc:374] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:38.831687 26653 raft_consensus.cc:729] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7d921da6534fd7bc9140df6f00d15e, State: Initialized, Role: FOLLOWER
I20260929 06:31:38.831842 26653 consensus_queue.cc:260] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:38.832126 26653 ts_tablet_manager.cc:1428] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:31:38.832264 26352 heartbeater.cc:503] Master 127.13.15.190:44107 was elected leader, sending a full tablet report...
Sep 29 06:31:38 dist-test-slave-sm0p krb5kdc[25158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663498, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for HTTP/127.13.15.148@KRBTEST.COM
W20260929 06:31:38.994587 26487 tablet.cc:2369] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:31:39.006762 26621 tablet.cc:2369] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:31:39.021170 26659 raft_consensus.cc:482] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:39.021324 26659 raft_consensus.cc:504] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:39.021790 26659 leader_election.cc:288] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f7d921da6534fd7bc9140df6f00d15e (127.13.15.129:38603), 14517eaf421746f9b1844283f25d28ee (127.13.15.130:34923)
I20260929 06:31:39.033223 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.022302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:58663 (local address 127.13.15.129:38603)
0929 06:31:39.022362 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:39.022364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:39.027356 (+ 4992us) server_negotiation.cc:407] Connection header received
0929 06:31:39.027470 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:39.027471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:39.027481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:39.027512 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:39.027753 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.028509 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.030121 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.030687 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.030739 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:39.030747 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:39.033084 (+ 2337us) server_negotiation.cc:299] Negotiation successful
0929 06:31:39.033146 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:31:39.033532 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.021936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38603 (local address 127.13.15.131:58663)
0929 06:31:39.027265 (+ 5329us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:39.027280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:39.027391 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:39.027584 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:39.027584 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:39.027586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:39.027668 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:39.027670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.028616 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.028617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:39.029920 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:39.029924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.032882 (+ 2958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.032884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:39.032957 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:39.032978 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 06:31:39.033397 (+ 419us) client_negotiation.cc:240] Negotiation successful
0929 06:31:39.033444 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5302,"thread_start_us":112,"threads_started":1}
I20260929 06:31:39.033965 26306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c82f09634824e6c9e9236cd32c044da" candidate_uuid: "f522f682d61a447197e588e5ecfcf5c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7d921da6534fd7bc9140df6f00d15e" is_pre_election: true
I20260929 06:31:39.034193 26306 raft_consensus.cc:2454] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f522f682d61a447197e588e5ecfcf5c9 in term 0.
I20260929 06:31:39.034475 26508 leader_election.cc:302] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7d921da6534fd7bc9140df6f00d15e, f522f682d61a447197e588e5ecfcf5c9; no voters:
I20260929 06:31:39.034586 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.022348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34923 (local address 127.13.15.131:60841)
0929 06:31:39.027842 (+ 5494us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:39.027855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:39.027954 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:39.028157 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:39.028158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:39.028160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:39.028227 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:39.028228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.029137 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.029138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:39.029850 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:39.029851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.034331 (+ 4480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.034332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:39.034410 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:39.034415 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:31:39.034473 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 06:31:39.034512 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5475,"thread_start_us":82,"threads_started":1}
I20260929 06:31:39.034623 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.022302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:60841 (local address 127.13.15.130:34923)
0929 06:31:39.022411 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:39.022413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:39.027925 (+ 5512us) server_negotiation.cc:407] Connection header received
0929 06:31:39.028029 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:39.028030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:39.028041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:39.028084 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:39.028308 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.029016 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.029941 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.030496 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.030590 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:39.030600 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:39.034491 (+ 3891us) server_negotiation.cc:299] Negotiation successful
0929 06:31:39.034537 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 06:31:39.034600 26659 raft_consensus.cc:2781] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:31:39.034685 26659 raft_consensus.cc:482] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:31:39.034715 26659 raft_consensus.cc:3037] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 0 FOLLOWER]: Advancing to term 1
W20260929 06:31:39.035395 26353 tablet.cc:2369] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:31:39.035686 26659 raft_consensus.cc:504] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:39.035817 26659 leader_election.cc:288] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [CANDIDATE]: Term 1 election: Requested vote from peers 1f7d921da6534fd7bc9140df6f00d15e (127.13.15.129:38603), 14517eaf421746f9b1844283f25d28ee (127.13.15.130:34923)
I20260929 06:31:39.036123 26430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c82f09634824e6c9e9236cd32c044da" candidate_uuid: "f522f682d61a447197e588e5ecfcf5c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14517eaf421746f9b1844283f25d28ee" is_pre_election: true
I20260929 06:31:39.036265 26430 raft_consensus.cc:2454] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f522f682d61a447197e588e5ecfcf5c9 in term 0.
I20260929 06:31:39.036530 26306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c82f09634824e6c9e9236cd32c044da" candidate_uuid: "f522f682d61a447197e588e5ecfcf5c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7d921da6534fd7bc9140df6f00d15e"
I20260929 06:31:39.036608 26306 raft_consensus.cc:3037] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:39.036731 26430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c82f09634824e6c9e9236cd32c044da" candidate_uuid: "f522f682d61a447197e588e5ecfcf5c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14517eaf421746f9b1844283f25d28ee"
I20260929 06:31:39.036801 26430 raft_consensus.cc:3037] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:39.037454 26306 raft_consensus.cc:2454] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f522f682d61a447197e588e5ecfcf5c9 in term 1.
I20260929 06:31:39.037647 26508 leader_election.cc:302] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7d921da6534fd7bc9140df6f00d15e, f522f682d61a447197e588e5ecfcf5c9; no voters:
I20260929 06:31:39.037712 26430 raft_consensus.cc:2454] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f522f682d61a447197e588e5ecfcf5c9 in term 1.
I20260929 06:31:39.037834 26659 raft_consensus.cc:2781] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:39.037982 26659 raft_consensus.cc:686] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 1 LEADER]: Becoming Leader. State: Replica: f522f682d61a447197e588e5ecfcf5c9, State: Running, Role: LEADER
I20260929 06:31:39.038074 26659 consensus_queue.cc:237] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } }
I20260929 06:31:39.038928 26153 catalog_manager.cc:5495] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f522f682d61a447197e588e5ecfcf5c9 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "f522f682d61a447197e588e5ecfcf5c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f522f682d61a447197e588e5ecfcf5c9" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 36931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 } health_report { overall_health: UNKNOWN } } }
I20260929 06:31:39.108356 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.104998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.13.15.131:36931)
0929 06:31:39.105074 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:39.105077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:39.105091 (+ 14us) server_negotiation.cc:407] Connection header received
0929 06:31:39.105143 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:39.105144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:39.105171 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:39.105288 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:31:39.105477 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.106065 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.107201 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:39.107505 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:39.107583 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:39.107800 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:31:39.107944 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:31:39.108018 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:39.108247 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 06:31:39.108270 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:31:39.123605 26430 raft_consensus.cc:1261] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 1 FOLLOWER]: Refusing update from remote peer f522f682d61a447197e588e5ecfcf5c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:39.123940 26306 raft_consensus.cc:1261] T 3c82f09634824e6c9e9236cd32c044da P 1f7d921da6534fd7bc9140df6f00d15e [term 1 FOLLOWER]: Refusing update from remote peer f522f682d61a447197e588e5ecfcf5c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:39.124392 26659 consensus_queue.cc:1035] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f7d921da6534fd7bc9140df6f00d15e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:39.124837 26659 consensus_queue.cc:1035] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14517eaf421746f9b1844283f25d28ee" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:39.136530 26672 mvcc.cc:205] Tried to move back new op lower bound from 7334557692406132736 to 7334557692060643328. Current Snapshot: MvccSnapshot[applied={T|T < 7334557692406132736}]
I20260929 06:31:39.137840 26670 mvcc.cc:205] Tried to move back new op lower bound from 7334557692406132736 to 7334557692060643328. Current Snapshot: MvccSnapshot[applied={T|T < 7334557692406132736}]
I20260929 06:31:39.140441 26153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:39.140550 26153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:39.141304 26153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c623b195661f4fb89e8bfd7035d1a75a: Sending DeleteTablet for 3 replicas of tablet 3c82f09634824e6c9e9236cd32c044da
I20260929 06:31:39.141775 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c82f09634824e6c9e9236cd32c044da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:39 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:54788
I20260929 06:31:39.142076 26420 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c82f09634824e6c9e9236cd32c044da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:39 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:47320
I20260929 06:31:39.142730 26679 tablet_replica.cc:330] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: stopping tablet replica
I20260929 06:31:39.142699 26554 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c82f09634824e6c9e9236cd32c044da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:39 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:46820
I20260929 06:31:39.143224 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26192
I20260929 06:31:39.142916 26679 raft_consensus.cc:2229] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:39.148061 26679 raft_consensus.cc:2258] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:39.148721 26679 ts_tablet_manager.cc:1905] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:39.148346 26680 tablet_replica.cc:330] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: stopping tablet replica
I20260929 06:31:39.150569 26679 ts_tablet_manager.cc:1918] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:39.150650 26679 log.cc:1196] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/3c82f09634824e6c9e9236cd32c044da
I20260929 06:31:39.150816 26680 raft_consensus.cc:2229] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:31:39.150997 26679 ts_tablet_manager.cc:1939] T 3c82f09634824e6c9e9236cd32c044da P 14517eaf421746f9b1844283f25d28ee: Deleting consensus metadata
I20260929 06:31:39.151042 26680 raft_consensus.cc:2258] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:39.151511 26680 ts_tablet_manager.cc:1905] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:39.151636 26141 catalog_manager.cc:4841] TS 14517eaf421746f9b1844283f25d28ee (127.13.15.130:34923): tablet 3c82f09634824e6c9e9236cd32c044da (table test-table [id=de7e07ba277244ec9eaf12c700f04b7d]) successfully deleted
I20260929 06:31:39.153301 26680 ts_tablet_manager.cc:1918] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:39.153430 26680 log.cc:1196] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/3c82f09634824e6c9e9236cd32c044da
I20260929 06:31:39.153759 26680 ts_tablet_manager.cc:1939] T 3c82f09634824e6c9e9236cd32c044da P f522f682d61a447197e588e5ecfcf5c9: Deleting consensus metadata
I20260929 06:31:39.154266 26140 catalog_manager.cc:4841] TS f522f682d61a447197e588e5ecfcf5c9 (127.13.15.131:36931): tablet 3c82f09634824e6c9e9236cd32c044da (table test-table [id=de7e07ba277244ec9eaf12c700f04b7d]) successfully deleted
W20260929 06:31:39.155884 26139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:38603 (error 108)
I20260929 06:31:39.156400 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:39.156292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38603 (local address 127.0.0.1:54792)
0929 06:31:39.156331 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:39.156364 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260929 06:31:39.156535 26139 catalog_manager.cc:4584] TS 1f7d921da6534fd7bc9140df6f00d15e (127.13.15.129:38603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c82f09634824e6c9e9236cd32c044da: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38603: connect: Connection refused (error 111)
I20260929 06:31:39.156862 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26356
I20260929 06:31:39.162361 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26489
I20260929 06:31:39.168700 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26121
I20260929 06:31:39.173769 13374 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 06:31:39.518263 13374 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 06:31:39.518354 13374 mini_ranger.cc:66] Stopping Ranger...
I20260929 06:31:39.853559 13374 mini_ranger.cc:68] Stopped Ranger
2026-09-29 06:31:39.853 UTC [25188] LOG: received smart shutdown request
2026-09-29 06:31:39.857 UTC [25188] LOG: background worker "logical replication launcher" (PID 25197) exited with exit code 1
2026-09-29 06:31:39.857 UTC [25192] LOG: shutting down
2026-09-29 06:31:40.022 UTC [25188] LOG: database system is shut down
2026-09-29T06:31:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47880 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:31:40 dist-test-slave-sm0p krb5kdc[26690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:31:40 dist-test-slave-sm0p krb5kdc[26690](info): set up 2 sockets
Sep 29 06:31:40 dist-test-slave-sm0p krb5kdc[26690](info): commencing operation
krb5kdc: starting...
W20260929 06:31:41.162436 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:31:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:31:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:31:41.274423 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:46005
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:41.353080 26703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:41.353263 26703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:41.353284 26703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:41.354346 26703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:41.354384 26703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:41.354398 26703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:41.354410 26703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:41.354440 26703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:41.355861 26703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:46005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:41.356035 26703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:41.356184 26703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:41.360949 26711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:41.361002 26708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:41.360977 26709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:41.361174 26703 server_base.cc:1034] running on GCE node
I20260929 06:31:41.361495 26703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:41.361727 26703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:41.362888 26703 hybrid_clock.cc:648] HybridClock initialized: now 1790663501362869 us; error 49 us; skew 500 ppm
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:41.365322 26703 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:41.365856 26703 webserver.cc:460] Webserver started at http://127.13.15.190:41575/ using document root <none> and password file <none>
I20260929 06:31:41.366190 26703 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:41.366255 26703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:41.366382 26703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:41.367390 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "d8f77712114540e69d561de7e54fc65e"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "1439c52f982c081157ac7738a7e933a8"
server_key_iv: "548beeb1cb1b9e9e4797f5f31fc57003"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.367754 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "d8f77712114540e69d561de7e54fc65e"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "1439c52f982c081157ac7738a7e933a8"
server_key_iv: "548beeb1cb1b9e9e4797f5f31fc57003"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.369012 26703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:31:41.369735 26718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:41.369928 26703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:31:41.370025 26703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "d8f77712114540e69d561de7e54fc65e"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "1439c52f982c081157ac7738a7e933a8"
server_key_iv: "548beeb1cb1b9e9e4797f5f31fc57003"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.370110 26703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:41.386921 26703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:41.389138 26703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:41.389288 26703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:41.393308 26703 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:46005
I20260929 06:31:41.393338 26770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:46005 every 8 connection(s)
I20260929 06:31:41.393672 26703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:41.394532 26771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:41.395969 26771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:31:41.398574 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26703
I20260929 06:31:41.398680 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:31:41.398869 13374 external_mini_cluster.cc:1428] Setting key 3e13ef05b206223b7d865d128dc31982
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:41.406199 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.396604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.13.15.190:46005)
0929 06:31:41.396758 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.396761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.396787 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:31:41.397280 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.397282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.397299 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.397379 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.397864 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.398940 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.399791 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.400249 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.400329 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.402826 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.402833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.402833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.402838 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.404686 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.405069 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.405073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.405073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.405182 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.405390 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.405391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.405391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.405681 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.405769 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.405932 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.406078 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260929 06:31:41.406204 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.396532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:46005 (local address 127.0.0.1:44008)
0929 06:31:41.396684 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.396724 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.397153 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.397404 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.397406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.397635 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.397781 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.397784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.399033 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.399034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.399697 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.399699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.400334 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.400335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.400419 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.400904 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.400914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.402744 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.404771 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.404777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.404779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.404980 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.405253 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.405254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.405254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.405329 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.405753 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.405757 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.405807 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.405876 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.405989 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260929 06:31:41.406996 26771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.15.189@KRBTEST.COM; defaulting to no policy
I20260929 06:31:41.407336 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.407238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46830)
0929 06:31:41.407270 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.407310 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260929 06:31:41.407460 26720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
I20260929 06:31:41.407804 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.407711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46832)
0929 06:31:41.407761 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.407783 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 06:31:41.407903 26771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:44473: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
I20260929 06:31:41.423152 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:44473
--webserver_interface=127.13.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:41.457139 26771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } attempt: 1
I20260929 06:31:41.457588 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.457458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46848)
0929 06:31:41.457522 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.457558 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 06:31:41.457952 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.457858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46852)
0929 06:31:41.457904 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.457928 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 06:31:41.458035 26771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:44473: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
W20260929 06:31:41.500901 26778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:41.501076 26778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:41.501096 26778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:41.502231 26778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:41.502269 26778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:41.502283 26778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:41.502295 26778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:41.502348 26778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:41.503847 26778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:41.504076 26778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:41.504304 26778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:41.509565 26783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:41.509624 26778 server_base.cc:1034] running on GCE node
W20260929 06:31:41.509531 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:41.509536 26786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:41.510162 26778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:41.510514 26778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:41.511683 26778 hybrid_clock.cc:648] HybridClock initialized: now 1790663501511661 us; error 35 us; skew 500 ppm
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:41.514199 26778 init.cc:377] Logged in from keytab as kudu/127.13.15.189@KRBTEST.COM (short username kudu)
I20260929 06:31:41.514631 26778 webserver.cc:460] Webserver started at http://127.13.15.189:40003/ using document root <none> and password file <none>
I20260929 06:31:41.514868 26778 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:41.514937 26778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:41.515077 26778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:41.516084 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data/instance:
uuid: "17effa7908ea47fda4252797a8f80ded"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "5d5b5fe656ebd6fc18cc4e4054ea89e7"
server_key_iv: "f82c26fd8f89521df839671872a337f9"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.516439 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal/instance:
uuid: "17effa7908ea47fda4252797a8f80ded"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "5d5b5fe656ebd6fc18cc4e4054ea89e7"
server_key_iv: "f82c26fd8f89521df839671872a337f9"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.517796 26778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:31:41.518579 26793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:41.518741 26778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:41.518836 26778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "17effa7908ea47fda4252797a8f80ded"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "5d5b5fe656ebd6fc18cc4e4054ea89e7"
server_key_iv: "f82c26fd8f89521df839671872a337f9"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.518916 26778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:41.526201 26771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } attempt: 2
I20260929 06:31:41.526624 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.526485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46868)
0929 06:31:41.526544 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.526589 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 06:31:41.526986 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.526893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46872)
0929 06:31:41.526947 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.526970 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 06:31:41.527083 26771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:44473: Network error: Client connection negotiation failed: client connection to 127.13.15.189:44473: connect: Connection refused (error 111)
I20260929 06:31:41.534780 26778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:41.536528 26778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:41.536633 26778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:41.540510 26778 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.189:44473
I20260929 06:31:41.540562 26845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.189:44473 every 8 connection(s)
I20260929 06:31:41.540859 26778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
I20260929 06:31:41.541721 26846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:41.543241 26846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:31:41.547343 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26778
I20260929 06:31:41.547462 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-1/wal/instance
I20260929 06:31:41.547685 13374 external_mini_cluster.cc:1428] Setting key 777175cc7cc1fcd632e6646a7ec0a3cd
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:41.551669 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.543881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.13.15.190:46005)
0929 06:31:41.543921 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.543923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.544108 (+ 185us) server_negotiation.cc:407] Connection header received
0929 06:31:41.544497 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.544498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.544509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.544567 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.545078 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.545580 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.546543 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.546739 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.546791 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.548888 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.548890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.548891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.548898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.550308 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.550779 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.550780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.550780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.550832 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.551105 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.551105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.551106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.551202 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.551250 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.551545 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.551610 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:31:41.551967 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.543854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:46005 (local address 127.0.0.1:44016)
0929 06:31:41.543994 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.544025 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.544425 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.544618 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.544620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.544842 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.545013 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.545016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.545653 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.545653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.546477 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.546479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.546808 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.546809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.546899 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.547307 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.547318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.548795 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.550391 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.550397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.550399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.550694 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.550900 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.550901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.550901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.551036 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.551264 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.551269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.551419 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.551529 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.551699 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260929 06:31:41.552620 26846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.15.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.188@KRBTEST.COM" created.
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:31:41.562158 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.552861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46880)
0929 06:31:41.552919 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.552927 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.552969 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.553259 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.553259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.553397 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.553437 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.553438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.554334 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.554335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.554829 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.554830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.555293 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.555294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.555358 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.555744 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.555749 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.559467 (+ 3718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.561143 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.561144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.561144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.561316 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.561519 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.561520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.561520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.561584 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.561841 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.561842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.561876 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.561945 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.562034 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 06:31:41.562343 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.552935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46880 (local address 127.13.15.189:44473)
0929 06:31:41.553100 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.553104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.553118 (+ 14us) server_negotiation.cc:407] Connection header received
0929 06:31:41.553134 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.553136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.553146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.553257 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.553526 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.554242 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.554923 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.555209 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.555294 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.559548 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.559555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.559556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.559562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.561034 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.561404 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.561407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.561407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.561458 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.561650 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.561651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.561651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.561777 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.561860 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.561974 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.562106 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260929 06:31:41.562881 26846 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:31:41.563208 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.563115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52314)
0929 06:31:41.563153 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.563190 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 06:31:41.563324 26796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
I20260929 06:31:41.563719 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.563636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52316)
0929 06:31:41.563676 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.563698 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:31:41.563813 26846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34523: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
I20260929 06:31:41.573716 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34523
--webserver_interface=127.13.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:36909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:41.613044 26846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } attempt: 1
I20260929 06:31:41.613580 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.613434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52322)
0929 06:31:41.613501 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.613540 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 06:31:41.613991 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.613905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52324)
0929 06:31:41.613950 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.613974 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 06:31:41.614107 26846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34523: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
I20260929 06:31:41.618203 26771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } attempt: 3
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:31:41.624807 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.618512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46890)
0929 06:31:41.618559 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.618570 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.618613 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.618773 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.618774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.618933 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.618993 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.618995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.619654 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.619656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.620147 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.620149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.620558 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.620560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.620648 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.621132 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.621137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.622349 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.623967 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.623968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.623968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.624121 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.624312 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.624312 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.624312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.624360 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.624599 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.624600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.624634 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.624681 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.624747 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 06:31:41.624859 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.618556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.13.15.189:44473)
0929 06:31:41.618605 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.618607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.618616 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:31:41.618677 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.618678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.618689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.618767 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.619083 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.619571 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.620227 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.620481 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.620567 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.622424 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.622426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.622426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.622433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.623889 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.624188 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.624189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.624189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.624248 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.624415 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.624416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.624416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.624541 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.624603 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.624703 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.624775 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 06:31:41.625403 26771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:31:41.625717 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.625622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52334)
0929 06:31:41.625666 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.625692 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 06:31:41.626044 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.625968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52340)
0929 06:31:41.626008 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.626030 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:31:41.626168 26771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34523: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
W20260929 06:31:41.654947 26853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:41.655129 26853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:41.655150 26853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:41.656149 26853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:41.656180 26853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:41.656194 26853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:41.656206 26853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:41.656217 26853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:41.657298 26771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } attempt: 1
I20260929 06:31:41.657670 26853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:46005,127.13.15.189:44473,127.13.15.188:34523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:41.657747 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.657604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52356)
0929 06:31:41.657661 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.657699 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 06:31:41.657905 26853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:41.658082 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.657989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52370)
0929 06:31:41.658032 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.658063 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:31:41.658186 26771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34523: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
I20260929 06:31:41.658195 26853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:41.662892 26858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:41.662972 26853 server_base.cc:1034] running on GCE node
W20260929 06:31:41.662892 26861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:41.662915 26859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:41.663384 26853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:41.663697 26853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:41.664832 26853 hybrid_clock.cc:648] HybridClock initialized: now 1790663501664820 us; error 26 us; skew 500 ppm
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:41.667291 26853 init.cc:377] Logged in from keytab as kudu/127.13.15.188@KRBTEST.COM (short username kudu)
I20260929 06:31:41.667714 26853 webserver.cc:460] Webserver started at http://127.13.15.188:33627/ using document root <none> and password file <none>
I20260929 06:31:41.667888 26853 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:41.667927 26853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:41.668011 26853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:41.668983 26853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data/instance:
uuid: "30daeb5750ab4869821353d5b09ae522"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "cb64cc83d10e8e51c9faf0fac9c344a7"
server_key_iv: "1453553d28bb32462af2ace53346533a"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.669277 26853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal/instance:
uuid: "30daeb5750ab4869821353d5b09ae522"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "cb64cc83d10e8e51c9faf0fac9c344a7"
server_key_iv: "1453553d28bb32462af2ace53346533a"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.670553 26853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:31:41.671228 26868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:41.671399 26853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:31:41.671480 26853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "30daeb5750ab4869821353d5b09ae522"
format_stamp: "Formatted at 2026-09-29 06:31:41 on dist-test-slave-sm0p"
server_key: "cb64cc83d10e8e51c9faf0fac9c344a7"
server_key_iv: "1453553d28bb32462af2ace53346533a"
server_key_version: "encryptionkey@0"
I20260929 06:31:41.671547 26853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:41.680945 26853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:41.682317 26846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } attempt: 2
I20260929 06:31:41.682713 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.682601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52386)
0929 06:31:41.682654 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.682688 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 06:31:41.683043 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.682948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52394)
0929 06:31:41.682991 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.683021 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 06:31:41.683135 26846 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34523: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34523: connect: Connection refused (error 111)
I20260929 06:31:41.683152 26853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:41.683298 26853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:41.687111 26853 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.188:34523
I20260929 06:31:41.687139 26920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.188:34523 every 8 connection(s)
I20260929 06:31:41.687485 26853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
I20260929 06:31:41.688081 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 26853
I20260929 06:31:41.688189 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790663323716959-13374-0/minicluster-data/master-2/wal/instance
I20260929 06:31:41.688409 13374 external_mini_cluster.cc:1428] Setting key e14ee6a9fb24a47be3d0dad0e3e96e8d
I20260929 06:31:41.688524 26921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:41.690618 26921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:41.699482 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.691255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.13.15.190:46005)
0929 06:31:41.691320 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.691322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.691455 (+ 133us) server_negotiation.cc:407] Connection header received
0929 06:31:41.691516 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.691517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.691525 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.691584 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.692091 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.692767 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.693750 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.693949 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.694036 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.696524 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.696526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.696526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.696534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.698199 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698658 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.698659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.698659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.698731 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698909 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.698910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.698910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.699048 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.699112 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.699311 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.699397 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:31:41.699687 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.688908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.13.15.188:34523)
0929 06:31:41.689105 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.689116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.689146 (+ 30us) server_negotiation.cc:407] Connection header received
0929 06:31:41.689665 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.689671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.689718 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.689865 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.690566 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.691735 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.692725 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.693313 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.693420 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.695861 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.695868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.695868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.695874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.697974 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698399 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.698401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.698401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.698502 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698729 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.698730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.698730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.699031 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.699168 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.699311 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.699420 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260929 06:31:41.699920 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.691205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:46005 (local address 127.0.0.1:44026)
0929 06:31:41.691353 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.691391 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.691456 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.691617 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.691619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.691934 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.692005 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.692007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.692851 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.692852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.693647 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.693650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.694046 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.694047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.694120 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.694642 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.694649 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.696421 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.698275 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.698284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.698581 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.698798 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.698798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.698799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.698848 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.699114 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.699121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.699182 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.699276 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.699822 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":419,"thread_start_us":94,"threads_started":1}
I20260929 06:31:41.700536 26921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:41.707145 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.700764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:44473 (local address 127.0.0.1:46900)
0929 06:31:41.700818 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.700888 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.700942 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.701069 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.701069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.701175 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.701227 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.701229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.701805 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.701806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.702371 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.702373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.702749 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.702750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.702814 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.703235 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.703239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.704445 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.706236 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.706237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.706238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.706442 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.706631 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.706632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.706632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.706698 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.706910 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.706911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.706954 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.707022 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.707084 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 06:31:41.707274 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.700856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.13.15.189:44473)
0929 06:31:41.700907 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.700909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.700936 (+ 27us) server_negotiation.cc:407] Connection header received
0929 06:31:41.700993 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.700994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.701005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.701069 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.701302 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.701731 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.702442 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.702668 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.702738 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.704559 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.704561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.704561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.704568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.706161 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.706520 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.706521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.706521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.706570 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.706758 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.706758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.706759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.706850 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.706921 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.707078 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.707156 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:31:41.707641 26921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:31:41.708274 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.700496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44034 (local address 127.13.15.190:46005)
0929 06:31:41.700527 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.700529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.700537 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:31:41.700571 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.700572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.700580 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.700631 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.701225 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.701738 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.702502 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.702664 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.702739 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.705154 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.705156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.705156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.705163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.706853 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.707259 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.707260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.707260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.707312 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.707570 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.707571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.707572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.707701 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.707900 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.708096 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.708178 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:31:41.714702 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.707869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52408)
0929 06:31:41.707952 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.707961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.708005 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.708129 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.708130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.708264 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.708299 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.708301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.709323 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.709323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.710043 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.710045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.710630 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.710631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.710669 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.711072 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.711077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.712110 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.713724 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.713725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.713726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.713884 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.714090 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.714091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.714091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.714168 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.714404 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.714404 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.714440 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.714508 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.714600 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 06:31:41.714757 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.707918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.13.15.188:34523)
0929 06:31:41.707953 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.707955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.707992 (+ 37us) server_negotiation.cc:407] Connection header received
0929 06:31:41.708061 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.708062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.708072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.708126 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.708532 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.709242 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.710129 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.710554 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.710625 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.712192 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.712194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.712195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.712202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.713631 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.713972 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.713973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.713974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.714026 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.714225 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.714226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.714227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.714338 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.714411 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.714533 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.714647 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":18}
I20260929 06:31:41.716053 26921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: Bootstrap starting.
I20260929 06:31:41.716514 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.708930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.13.15.189:44473)
0929 06:31:41.708962 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.708963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.708971 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:31:41.709036 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.709036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.709044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.709104 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.709512 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.710234 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.711011 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.711206 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.711286 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.713708 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.713709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.713710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.713717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.715423 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.715757 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.715758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.715758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.715811 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.716009 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.716010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.716010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.716137 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.716323 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.716358 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.716426 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:31:41.717085 26921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:41.717427 26921 log.cc:824] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:41.718236 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:31:41.718271 26921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: No bootstrap required, opened a new log
I20260929 06:31:41.720037 26921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.720235 26921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:41.720333 26921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30daeb5750ab4869821353d5b09ae522, State: Initialized, Role: FOLLOWER
I20260929 06:31:41.720518 26921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.720854 26921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:41.722965 26934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.723065 26934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:31:41.723258 26945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:31:41.723313 26945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:31:41.722883 26853 master_runner.cc:186] Error getting master registration for 127.13.15.190:46005: OK, instance_id { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" instance_seqno: 1790663501392167 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:31:41.724522 26853 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:31:41.729209 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.719562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44048 (local address 127.13.15.190:46005)
0929 06:31:41.719602 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.719604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.719932 (+ 328us) server_negotiation.cc:407] Connection header received
0929 06:31:41.720006 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.720007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.720018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.720050 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.720970 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.721648 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.722646 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.722921 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.722990 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.726311 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.726312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.726312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.726334 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.728013 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.728432 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.728433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.728433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.728483 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.728658 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.728659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.728659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.728766 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.728840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.729056 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.729127 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:31:41.729310 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.719680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.13.15.188:34523)
0929 06:31:41.719716 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.719717 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.719794 (+ 77us) server_negotiation.cc:407] Connection header received
0929 06:31:41.719835 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.719835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.719844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.719912 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.720713 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.721377 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.722516 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.722780 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.722817 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.726268 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.726269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.726270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.726276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.727904 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.728290 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.728291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.728291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.728362 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.728575 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.728576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.728577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.728720 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.728801 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.729165 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.729228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:31:41.730711 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.719595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.13.15.189:44473)
0929 06:31:41.719634 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.719635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.720072 (+ 437us) server_negotiation.cc:407] Connection header received
0929 06:31:41.720163 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.720164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.720174 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.720202 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.720889 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.721583 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.724796 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.725616 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.725665 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.727968 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.727969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.727969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.727976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.729563 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.729917 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.729918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.729919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.729977 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.730222 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.730223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.730224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.730353 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.730462 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.730535 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.730607 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:31:41.733316 26771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } attempt: 2
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:31:41.739806 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.733557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52412)
0929 06:31:41.733676 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.733686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.733738 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.733850 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.733850 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.734007 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.734044 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.734046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.734651 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.734652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.735415 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.735417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.735700 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.735701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.735770 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.736091 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.736096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.737415 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.739010 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.739011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.739011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.739160 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.739339 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.739339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.739339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.739384 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.739614 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.739615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.739645 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.739692 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.739751 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260929 06:31:41.739856 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.733629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.13.15.188:34523)
0929 06:31:41.733670 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.733672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.733723 (+ 51us) server_negotiation.cc:407] Connection header received
0929 06:31:41.733792 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.733793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.733800 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.733839 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.734107 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.734577 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.735482 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.735638 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.735680 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.737495 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.737497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.737497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.737504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.738919 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.739223 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.739223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.739224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.739278 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.739435 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.739436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.739436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.739557 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.739617 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.739712 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.739778 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:31:41.741694 26771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: Bootstrap starting.
I20260929 06:31:41.742439 26771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:41.742733 26771 log.cc:824] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:41.743531 26771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: No bootstrap required, opened a new log
I20260929 06:31:41.744858 26771 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.744980 26771 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:41.745048 26771 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f77712114540e69d561de7e54fc65e, State: Initialized, Role: FOLLOWER
I20260929 06:31:41.745224 26771 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.745504 26948 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.745573 26771 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:41.745581 26948 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:31:41.747629 26959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:31:41.747722 26959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:31:41.748193 26703 master_runner.cc:186] Error getting master registration for 127.13.15.189:44473: OK, instance_id { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" instance_seqno: 1790663501539390 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:31:41.750473 26703 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:31:41.774261 26846 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } attempt: 3
Sep 29 06:31:41 dist-test-slave-sm0p krb5kdc[26690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:31:41.783022 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.774525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34523 (local address 127.0.0.1:52416)
0929 06:31:41.774572 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:41.774583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:41.774618 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:41.774793 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:41.774794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:41.774974 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:41.775056 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.775057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.775949 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.775950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.776737 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:41.776739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.777222 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.777223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:41.777282 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.777787 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:41.777793 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:41.779368 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:41.781180 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.781181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.781182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.781482 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.781747 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:41.781748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:41.781748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:41.781852 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:41.782694 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:41.782695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:41.782742 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 06:31:41.782815 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 06:31:41.782910 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 06:31:41.783027 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:41.774597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.13.15.188:34523)
0929 06:31:41.774675 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:41.774677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:41.774685 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:31:41.774706 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:41.774707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:41.774717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:41.774777 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:41.775127 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.775865 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.776822 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:41.777120 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:41.777209 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:41.779454 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:41.779456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:41.779456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:41.779463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:41.781098 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.781580 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.781581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.781581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.781658 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:41.781966 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:41.781968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:41.781968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:41.782171 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:41.782281 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:41.782855 (+ 574us) server_negotiation.cc:299] Negotiation successful
0929 06:31:41.782935 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 06:31:41.784940 26846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: Bootstrap starting.
I20260929 06:31:41.785689 26846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:41.785993 26846 log.cc:824] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:41.786877 26846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: No bootstrap required, opened a new log
I20260929 06:31:41.788949 26846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.789093 26846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:41.789129 26846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17effa7908ea47fda4252797a8f80ded, State: Initialized, Role: FOLLOWER
I20260929 06:31:41.789304 26846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.789661 26846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:41.789624 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.791231 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:31:41.791544 26973 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:31:41.791622 26973 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:31:41.794570 26778 master_runner.cc:263] No leader master found from master 17effa7908ea47fda4252797a8f80ded
I20260929 06:31:41.794615 26778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:31:41.814062 26962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:41.814312 26962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.814836 26745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17effa7908ea47fda4252797a8f80ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f77712114540e69d561de7e54fc65e" is_pre_election: true
I20260929 06:31:41.814996 26745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17effa7908ea47fda4252797a8f80ded in term 0.
I20260929 06:31:41.815290 26797 leader_election.cc:302] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17effa7908ea47fda4252797a8f80ded, d8f77712114540e69d561de7e54fc65e; no voters:
I20260929 06:31:41.815835 26895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17effa7908ea47fda4252797a8f80ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30daeb5750ab4869821353d5b09ae522" is_pre_election: true
I20260929 06:31:41.815977 26895 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17effa7908ea47fda4252797a8f80ded in term 0.
I20260929 06:31:41.816260 26962 leader_election.cc:288] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8f77712114540e69d561de7e54fc65e (127.13.15.190:46005), 30daeb5750ab4869821353d5b09ae522 (127.13.15.188:34523)
I20260929 06:31:41.816430 26962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:31:41.816521 26962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:31:41.816593 26962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:41.817634 26962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.817962 26745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17effa7908ea47fda4252797a8f80ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f77712114540e69d561de7e54fc65e"
I20260929 06:31:41.818063 26745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:41.818919 26745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17effa7908ea47fda4252797a8f80ded in term 1.
I20260929 06:31:41.819228 26797 leader_election.cc:302] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17effa7908ea47fda4252797a8f80ded, d8f77712114540e69d561de7e54fc65e; no voters:
I20260929 06:31:41.819512 26895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17effa7908ea47fda4252797a8f80ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30daeb5750ab4869821353d5b09ae522"
I20260929 06:31:41.819595 26895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:41.820474 26895 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17effa7908ea47fda4252797a8f80ded in term 1.
I20260929 06:31:41.820716 26962 leader_election.cc:288] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [CANDIDATE]: Term 1 election: Requested vote from peers d8f77712114540e69d561de7e54fc65e (127.13.15.190:46005), 30daeb5750ab4869821353d5b09ae522 (127.13.15.188:34523)
I20260929 06:31:41.820829 26962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:41.820919 26962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [term 1 LEADER]: Becoming Leader. State: Replica: 17effa7908ea47fda4252797a8f80ded, State: Running, Role: LEADER
I20260929 06:31:41.821046 26962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } }
I20260929 06:31:41.821725 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17effa7908ea47fda4252797a8f80ded. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.821900 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:41.822820 26977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:41.823081 26977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:31:41.824703 26895 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [term 1 FOLLOWER]: Refusing update from remote peer 17effa7908ea47fda4252797a8f80ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:41.825086 26975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:41.826536 26934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17effa7908ea47fda4252797a8f80ded. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.826848 26934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:31:41.827293 26745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [term 1 FOLLOWER]: Refusing update from remote peer 17effa7908ea47fda4252797a8f80ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:41.828658 26975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:41.829881 26948 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17effa7908ea47fda4252797a8f80ded. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.829972 26948 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:31:41.831290 26975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.831462 26975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:41.832020 26934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.832113 26934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:31:41.832355 26978 mvcc.cc:205] Tried to move back new op lower bound from 7334557703471525888 to 7334557703460708352. Current Snapshot: MvccSnapshot[applied={T|T < 7334557703471525888}]
I20260929 06:31:41.832546 26979 mvcc.cc:205] Tried to move back new op lower bound from 7334557703471525888 to 7334557703460708352. Current Snapshot: MvccSnapshot[applied={T|T < 7334557703471525888}]
I20260929 06:31:41.833150 26977 catalog_manager.cc:1347] Generated new cluster ID: 15c33beb995146b88e71f690a5f0848e
I20260929 06:31:41.833196 26977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:31:41.845484 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.845588 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:41.846295 26948 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17effa7908ea47fda4252797a8f80ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8f77712114540e69d561de7e54fc65e" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 46005 } } peers { permanent_uuid: "17effa7908ea47fda4252797a8f80ded" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 44473 } } peers { permanent_uuid: "30daeb5750ab4869821353d5b09ae522" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34523 } } }
I20260929 06:31:41.846410 26948 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:31:41.846839 26981 mvcc.cc:205] Tried to move back new op lower bound from 7334557703471525888 to 7334557703460708352. Current Snapshot: MvccSnapshot[applied={T|T < 7334557703471525888}]
I20260929 06:31:41.865402 26977 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:31:41.866227 26977 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:41.875177 26977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17effa7908ea47fda4252797a8f80ded: Generated new TSK 0
I20260929 06:31:41.875485 26977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:31:42.723747 26945 catalog_manager.cc:1259] Loaded cluster ID: 15c33beb995146b88e71f690a5f0848e
I20260929 06:31:42.723841 26945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: loading cluster ID for follower catalog manager: success
I20260929 06:31:42.724622 26945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: acquiring CA information for follower catalog manager: success
I20260929 06:31:42.724905 26945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30daeb5750ab4869821353d5b09ae522: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:31:42.748186 26959 catalog_manager.cc:1259] Loaded cluster ID: 15c33beb995146b88e71f690a5f0848e
I20260929 06:31:42.748261 26959 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: loading cluster ID for follower catalog manager: success
I20260929 06:31:42.749092 26959 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: acquiring CA information for follower catalog manager: success
I20260929 06:31:42.749408 26959 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8f77712114540e69d561de7e54fc65e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:31:42.925980 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26703
I20260929 06:31:42.931321 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26778
I20260929 06:31:42.936012 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 26853
2026-09-29T06:31:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2806 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:31:42 dist-test-slave-sm0p krb5kdc[26993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:31:42 dist-test-slave-sm0p krb5kdc[26993](info): set up 2 sockets
Sep 29 06:31:42 dist-test-slave-sm0p krb5kdc[26993](info): commencing operation
krb5kdc: starting...
W20260929 06:31:43.966323 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[26993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:31:44.078888 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34427
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:44.158030 27006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:44.158236 27006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:44.158257 27006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:44.159318 27006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:44.159358 27006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:44.159370 27006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:44.159381 27006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:44.159392 27006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:44.160749 27006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:44.160974 27006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:44.161203 27006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:44.165850 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:44.165858 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:44.166066 27006 server_base.cc:1034] running on GCE node
W20260929 06:31:44.165966 27014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:44.166329 27006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:44.166601 27006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:44.167739 27006 hybrid_clock.cc:648] HybridClock initialized: now 1790663504167720 us; error 31 us; skew 500 ppm
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[26993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663504, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:44.170076 27006 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:44.170500 27006 webserver.cc:460] Webserver started at http://127.13.15.190:37901/ using document root <none> and password file <none>
I20260929 06:31:44.170665 27006 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:44.170701 27006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:44.170782 27006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:44.171768 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "f158934578dd482b9c24c7613cad3873"
format_stamp: "Formatted at 2026-09-29 06:31:44 on dist-test-slave-sm0p"
server_key: "d6bd7fd08c18839285628a6fbcbc89f8"
server_key_iv: "85d405ce4d2b477d5a5df35bcfaf1901"
server_key_version: "encryptionkey@0"
I20260929 06:31:44.172075 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "f158934578dd482b9c24c7613cad3873"
format_stamp: "Formatted at 2026-09-29 06:31:44 on dist-test-slave-sm0p"
server_key: "d6bd7fd08c18839285628a6fbcbc89f8"
server_key_iv: "85d405ce4d2b477d5a5df35bcfaf1901"
server_key_version: "encryptionkey@0"
I20260929 06:31:44.173374 27006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:31:44.174019 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:44.174213 27006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:31:44.174315 27006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "f158934578dd482b9c24c7613cad3873"
format_stamp: "Formatted at 2026-09-29 06:31:44 on dist-test-slave-sm0p"
server_key: "d6bd7fd08c18839285628a6fbcbc89f8"
server_key_iv: "85d405ce4d2b477d5a5df35bcfaf1901"
server_key_version: "encryptionkey@0"
I20260929 06:31:44.174404 27006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:44.184201 27006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:44.185863 27006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:44.186010 27006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:44.189700 27006 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34427
I20260929 06:31:44.189728 27073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34427 every 8 connection(s)
I20260929 06:31:44.190045 27006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:44.190862 27074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:44.192943 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873: Bootstrap starting.
I20260929 06:31:44.193171 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27006
I20260929 06:31:44.193313 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:31:44.193521 13374 external_mini_cluster.cc:1428] Setting key fc9755faa632a9b8af48a0459696a3d2
I20260929 06:31:44.193660 27074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:44.194023 27074 log.cc:824] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:44.194780 27074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873: No bootstrap required, opened a new log
I20260929 06:31:44.197016 27074 raft_consensus.cc:348] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } }
I20260929 06:31:44.197172 27074 raft_consensus.cc:374] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:44.197218 27074 raft_consensus.cc:729] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f158934578dd482b9c24c7613cad3873, State: Initialized, Role: FOLLOWER
I20260929 06:31:44.197373 27074 consensus_queue.cc:260] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } }
I20260929 06:31:44.197456 27074 raft_consensus.cc:388] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:44.197497 27074 raft_consensus.cc:482] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:44.197544 27074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:44.198074 27074 raft_consensus.cc:504] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } }
I20260929 06:31:44.198192 27074 leader_election.cc:302] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f158934578dd482b9c24c7613cad3873; no voters:
I20260929 06:31:44.198338 27074 leader_election.cc:288] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:31:44.198401 27079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:44.198583 27074 sys_catalog.cc:564] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:44.198589 27079 raft_consensus.cc:686] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [term 1 LEADER]: Becoming Leader. State: Replica: f158934578dd482b9c24c7613cad3873, State: Running, Role: LEADER
I20260929 06:31:44.198768 27079 consensus_queue.cc:237] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } }
I20260929 06:31:44.199184 27081 sys_catalog.cc:455] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f158934578dd482b9c24c7613cad3873. Latest consensus state: current_term: 1 leader_uuid: "f158934578dd482b9c24c7613cad3873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } } }
I20260929 06:31:44.199261 27081 sys_catalog.cc:458] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:44.199638 27080 sys_catalog.cc:455] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f158934578dd482b9c24c7613cad3873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f158934578dd482b9c24c7613cad3873" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34427 } } }
I20260929 06:31:44.199708 27080 sys_catalog.cc:458] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:44.199882 27088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:44.200429 27088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[26993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:44.201967 27088 catalog_manager.cc:1347] Generated new cluster ID: 911fd4ccec6545509aaa2e02abab1feb
I20260929 06:31:44.202019 27088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:31:44.205017 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:44.194029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.13.15.190:34427)
0929 06:31:44.194271 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:44.194279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:44.194310 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:31:44.194896 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:44.194906 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:44.194975 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:44.195113 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:44.195676 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:44.197101 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:44.198083 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:44.198726 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:44.200206 (+ 1480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:44.201744 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:44.201747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:44.201748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:44.201759 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:44.203517 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:44.203899 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:44.203902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:44.203903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:44.203992 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:44.204198 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:44.204199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:44.204200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:44.204424 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:44.204595 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:44.204690 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 06:31:44.204825 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260929 06:31:44.205931 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:31:44.213080 27088 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:31:44.213574 27088 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:44.222115 27088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f158934578dd482b9c24c7613cad3873: Generated new TSK 0
I20260929 06:31:44.222317 27088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:31:44.259814 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27006
2026-09-29T06:31:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1322 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[27101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[27101](info): set up 2 sockets
Sep 29 06:31:44 dist-test-slave-sm0p krb5kdc[27101](info): commencing operation
krb5kdc: starting...
W20260929 06:31:45.298166 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:31:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:31:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:31:45.409726 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:45.486307 27114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:45.486552 27114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:45.486585 27114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:45.487715 27114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:45.487779 27114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:45.487802 27114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:45.487824 27114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:45.487852 27114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:45.489223 27114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:45.489483 27114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:45.489687 27114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:45.494263 27122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:45.494428 27120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:45.494314 27114 server_base.cc:1034] running on GCE node
W20260929 06:31:45.494272 27119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:45.494720 27114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:45.494942 27114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:45.496096 27114 hybrid_clock.cc:648] HybridClock initialized: now 1790663505496085 us; error 27 us; skew 500 ppm
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:45.498509 27114 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:45.498942 27114 webserver.cc:460] Webserver started at http://127.13.15.190:37403/ using document root <none> and password file <none>
I20260929 06:31:45.499148 27114 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:45.499191 27114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:45.499317 27114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:45.500290 27114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "369bc7159509454fbd4c92767e8c9606"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "1b3bd0dc60ad38f45211d9015fe022d5"
server_key_iv: "b848612d0fefb84a0798a415ff4b3ed7"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.500631 27114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "369bc7159509454fbd4c92767e8c9606"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "1b3bd0dc60ad38f45211d9015fe022d5"
server_key_iv: "b848612d0fefb84a0798a415ff4b3ed7"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.501861 27114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:31:45.502624 27129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.502812 27114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.502897 27114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "369bc7159509454fbd4c92767e8c9606"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "1b3bd0dc60ad38f45211d9015fe022d5"
server_key_iv: "b848612d0fefb84a0798a415ff4b3ed7"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.502969 27114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:45.527849 27114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:45.529671 27114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:45.529827 27114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:45.533900 27114 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44701
I20260929 06:31:45.533998 27181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44701 every 8 connection(s)
I20260929 06:31:45.534353 27114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:45.535310 27182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:45.537513 27182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap starting.
I20260929 06:31:45.538079 27182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:45.538376 27182 log.cc:824] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:45.539047 27182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: No bootstrap required, opened a new log
I20260929 06:31:45.540614 27182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:45.540745 27182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:45.540768 27182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Initialized, Role: FOLLOWER
I20260929 06:31:45.540957 27182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:45.541038 27182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:45.541078 27182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:45.541128 27182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:45.541901 27182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:45.542011 27182 leader_election.cc:302] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369bc7159509454fbd4c92767e8c9606; no voters:
I20260929 06:31:45.542205 27182 leader_election.cc:288] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:31:45.542258 27185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:45.542405 27185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 LEADER]: Becoming Leader. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Running, Role: LEADER
I20260929 06:31:45.542450 27182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:45.542541 27185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:45.542944 27186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:45.542968 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369bc7159509454fbd4c92767e8c9606. Latest consensus state: current_term: 1 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:45.543085 27186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:45.543111 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:45.543507 27193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:45.544061 27193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:31:45.544301 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27114
I20260929 06:31:45.544406 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:31:45.544618 13374 external_mini_cluster.cc:1428] Setting key 3111faf64a8712de783bf32b75ca08ff
I20260929 06:31:45.545702 27193 catalog_manager.cc:1347] Generated new cluster ID: 8ba1c81da48144d3aec2649aa2a7ea1d
I20260929 06:31:45.545754 27193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:45.554593 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:45.545174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.13.15.190:44701)
0929 06:31:45.545368 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:45.545378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:45.545439 (+ 61us) server_negotiation.cc:407] Connection header received
0929 06:31:45.545634 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:45.545642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:45.545697 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:45.545835 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:45.546576 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.547487 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.548226 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.548655 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.548780 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:45.551298 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:45.551301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:45.551302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:45.551313 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:45.552976 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.553351 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.553354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.553355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.553462 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.553670 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.553671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.553672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.553906 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:45.554077 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:45.554203 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 06:31:45.554352 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260929 06:31:45.587153 27193 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:31:45.587708 27193 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:45.592226 27193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Generated new TSK 0
I20260929 06:31:45.592404 27193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:31:45.623013 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:45.711844 27208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:45.712034 27208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:45.712057 27208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:45.713179 27208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:45.713217 27208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:45.713243 27208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:31:45.714599 27208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:45.714788 27208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:45.714996 27208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:45.719889 27214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:45.719983 27208 server_base.cc:1034] running on GCE node
W20260929 06:31:45.719940 27213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:45.720077 27216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:45.720311 27208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:45.720547 27208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:45.721719 27208 hybrid_clock.cc:648] HybridClock initialized: now 1790663505721704 us; error 25 us; skew 500 ppm
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:45.724048 27208 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:31:45.724452 27208 webserver.cc:460] Webserver started at http://127.13.15.129:35475/ using document root <none> and password file <none>
I20260929 06:31:45.724663 27208 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:45.724730 27208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:45.724874 27208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:45.725896 27208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "03942114d8bc446f85813895870f4041"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "45b63f32120b60c3f5b85e13d388be28"
server_key_iv: "6695174ed12b38af0827d5c559318391"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.726290 27208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "03942114d8bc446f85813895870f4041"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "45b63f32120b60c3f5b85e13d388be28"
server_key_iv: "6695174ed12b38af0827d5c559318391"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.727528 27208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:45.728243 27223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.728402 27208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:31:45.728511 27208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "03942114d8bc446f85813895870f4041"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "45b63f32120b60c3f5b85e13d388be28"
server_key_iv: "6695174ed12b38af0827d5c559318391"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.728590 27208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:45.740967 27208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:45.742933 27208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:45.743088 27208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:45.743311 27208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:45.743639 27208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:45.743693 27208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.743734 27208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:45.743762 27208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.749557 27208 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:41461
I20260929 06:31:45.749639 27336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:41461 every 8 connection(s)
I20260929 06:31:45.749943 27208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:45.758343 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27208
I20260929 06:31:45.758477 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:31:45.758677 13374 external_mini_cluster.cc:1428] Setting key 6f9c151838214ae9df927439f9a29402
I20260929 06:31:45.759477 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:45.750733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:55263 (local address 127.13.15.190:44701)
0929 06:31:45.750789 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:45.750792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:45.750987 (+ 195us) server_negotiation.cc:407] Connection header received
0929 06:31:45.751551 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:45.751552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:45.751567 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:45.751617 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:45.752546 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.753061 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.754042 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.754317 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.754401 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:45.756491 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:45.756492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:45.756492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:45.756499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:45.757913 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.758391 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.758393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.758393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.758465 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.758709 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.758709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.758710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.758840 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:45.758929 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:45.759254 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 06:31:45.759369 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 06:31:45.759653 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:45.750721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:55263)
0929 06:31:45.750875 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:45.750911 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:45.751411 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:45.751690 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:45.751693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:45.752254 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:45.752447 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:45.752451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.753165 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.753167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:45.753953 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:45.753956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.754416 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.754417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:45.754537 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:45.755072 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:45.755080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:45.756377 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:45.758015 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:45.758022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:45.758024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:45.758318 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:45.758515 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:45.758516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:45.758516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:45.758636 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:45.758922 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:45.758928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:45.759088 (+ 160us) client_negotiation.cc:769] Sending connection context
0929 06:31:45.759207 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 06:31:45.759397 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260929 06:31:45.760632 27337 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:45.760730 27337 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:45.760957 27337 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:45.761528 27146 ts_manager.cc:194] Registered new tserver with Master: 03942114d8bc446f85813895870f4041 (127.13.15.129:41461)
I20260929 06:31:45.762358 27146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:55263
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:31:45.784632 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:45.866483 27342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:45.866714 27342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:45.866753 27342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:45.867908 27342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:45.867972 27342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:45.868018 27342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:31:45.869495 27342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:45.869750 27342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:45.869975 27342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:45.875106 27348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:45.875164 27350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:45.875192 27347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:45.875229 27342 server_base.cc:1034] running on GCE node
I20260929 06:31:45.875587 27342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:45.875821 27342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:45.876963 27342 hybrid_clock.cc:648] HybridClock initialized: now 1790663505876953 us; error 31 us; skew 500 ppm
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:45.879477 27342 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:31:45.879905 27342 webserver.cc:460] Webserver started at http://127.13.15.130:36509/ using document root <none> and password file <none>
I20260929 06:31:45.880126 27342 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:45.880192 27342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:45.880342 27342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:45.881376 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "0d907ca2ed68447a949206ecfa95f974"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "e2af4e1df37b9bb4c3560f08e28f5976"
server_key_iv: "c093e8c8d30470788a4c3c70924d2609"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.881731 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "0d907ca2ed68447a949206ecfa95f974"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "e2af4e1df37b9bb4c3560f08e28f5976"
server_key_iv: "c093e8c8d30470788a4c3c70924d2609"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.883080 27342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:31:45.883754 27357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.883917 27342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.884019 27342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "0d907ca2ed68447a949206ecfa95f974"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "e2af4e1df37b9bb4c3560f08e28f5976"
server_key_iv: "c093e8c8d30470788a4c3c70924d2609"
server_key_version: "encryptionkey@0"
I20260929 06:31:45.884089 27342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:45.895929 27342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:45.898582 27342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:45.898732 27342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:45.898972 27342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:45.899282 27342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:45.899317 27342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.899338 27342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:45.899394 27342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:45.905092 27342 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43671
I20260929 06:31:45.905179 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43671 every 8 connection(s)
I20260929 06:31:45.905503 27342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:31:45.909157 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27342
I20260929 06:31:45.909284 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:31:45.909523 13374 external_mini_cluster.cc:1428] Setting key c8856437d951b19ee97c2522c8a5735c
Sep 29 06:31:45 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663505, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:45.915055 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:45.906296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:33471 (local address 127.13.15.190:44701)
0929 06:31:45.906370 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:45.906373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:45.906590 (+ 217us) server_negotiation.cc:407] Connection header received
0929 06:31:45.907055 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:45.907056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:45.907066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:45.907118 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:45.908002 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.908489 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.909529 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.909826 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.909898 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:45.912117 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:45.912119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:45.912119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:45.912126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:45.913582 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.914007 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.914008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.914009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.914075 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:45.914382 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:45.914383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:45.914383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:45.914510 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:45.914587 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:45.914865 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 06:31:45.914965 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:31:45.915241 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:45.906281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:33471)
0929 06:31:45.906474 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:45.906510 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:45.906976 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:45.907173 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:45.907176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:45.907730 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:45.907929 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:45.907934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.908565 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.908566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:45.909445 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:45.909447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:45.909920 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:45.909921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:45.910062 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:45.910672 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:45.910681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:45.912015 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:45.913687 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:45.913694 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:45.913697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:45.913927 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:45.914192 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:45.914193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:45.914194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:45.914309 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:45.914578 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:45.914585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:45.914727 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 06:31:45.914831 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 06:31:45.914993 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260929 06:31:45.915777 27471 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:45.915859 27471 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:45.916069 27471 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:45.916534 27146 ts_manager.cc:194] Registered new tserver with Master: 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671)
I20260929 06:31:45.917078 27146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:33471
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:31:45.936704 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:46.017230 27476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:46.017454 27476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:46.017477 27476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:46.018715 27476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:46.018754 27476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:46.018781 27476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:31:46.020129 27476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:46.020301 27476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:46.020471 27476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:46.025435 27484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.025532 27481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.025633 27476 server_base.cc:1034] running on GCE node
W20260929 06:31:46.025444 27482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.025911 27476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:46.026098 27476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:46.027273 27476 hybrid_clock.cc:648] HybridClock initialized: now 1790663506027260 us; error 26 us; skew 500 ppm
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:46.029642 27476 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:31:46.030085 27476 webserver.cc:460] Webserver started at http://127.13.15.131:44609/ using document root <none> and password file <none>
I20260929 06:31:46.030325 27476 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:46.030403 27476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:46.030550 27476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:31:46.031706 27476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "74546824567a43fca8dfda381c856ef4"
format_stamp: "Formatted at 2026-09-29 06:31:46 on dist-test-slave-sm0p"
server_key: "647c7881728aea89f7cf334df9ba7f81"
server_key_iv: "884868f257ede6273352f3534e432617"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.032050 27476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "74546824567a43fca8dfda381c856ef4"
format_stamp: "Formatted at 2026-09-29 06:31:46 on dist-test-slave-sm0p"
server_key: "647c7881728aea89f7cf334df9ba7f81"
server_key_iv: "884868f257ede6273352f3534e432617"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.033269 27476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:31:46.033950 27491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.034108 27476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:31:46.034226 27476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "74546824567a43fca8dfda381c856ef4"
format_stamp: "Formatted at 2026-09-29 06:31:46 on dist-test-slave-sm0p"
server_key: "647c7881728aea89f7cf334df9ba7f81"
server_key_iv: "884868f257ede6273352f3534e432617"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.034307 27476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:46.047962 27476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:46.049674 27476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:46.049831 27476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:46.050058 27476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:46.050426 27476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:46.050486 27476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.050529 27476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:46.050556 27476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.056510 27476 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40143
I20260929 06:31:46.056556 27604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40143 every 8 connection(s)
I20260929 06:31:46.056846 27476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:31:46.061579 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27476
I20260929 06:31:46.061684 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:31:46.061879 13374 external_mini_cluster.cc:1428] Setting key 4e5652ab58a0c0a3dde51967d39055ab
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:46.065891 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.057506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:37127 (local address 127.13.15.190:44701)
0929 06:31:46.057569 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.057571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.057747 (+ 176us) server_negotiation.cc:407] Connection header received
0929 06:31:46.058247 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.058248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.058258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.058318 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.059184 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.059649 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.060586 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.060759 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.060813 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.063403 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.063404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.063405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.063411 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.064848 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.065186 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.065187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.065187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.065238 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.065408 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.065408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.065408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.065516 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.065566 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.065770 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.065831 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:31:46.066124 27607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.057468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:37127)
0929 06:31:46.057621 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.057662 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.058122 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.058378 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.058381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.058914 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:46.059111 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.059116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.059722 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.059723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.060505 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.060507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.060831 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.060831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.060927 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.061531 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:46.061542 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:46.063318 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:46.064915 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.064919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.064921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.065125 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.065278 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.065278 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.065279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.065358 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.065570 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:46.065574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:46.065678 (+ 104us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.065756 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.065876 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260929 06:31:46.066634 27605 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:46.066727 27605 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:46.066869 27605 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:46.067179 27146 ts_manager.cc:194] Registered new tserver with Master: 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.067512 27146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:37127
I20260929 06:31:46.073560 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:31:46.080955 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.074956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.13.15.190:44701)
0929 06:31:46.075018 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.075020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.075147 (+ 127us) server_negotiation.cc:407] Connection header received
0929 06:31:46.075183 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.075184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.075192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.075232 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.075820 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.076296 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.077013 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.077199 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.077290 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.078548 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.078549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.078550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.078556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.079769 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.080120 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.080121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.080121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.080178 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.080377 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.080378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.080379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.080526 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.080628 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.080752 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.080831 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:31:46.082509 27146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:31:46.084054 27146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:31:46.090785 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43671 (local address 127.0.0.1:53850)
0929 06:31:46.086482 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.086524 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.086593 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.086898 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.086901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.086903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.086978 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.086979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088469 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.088470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.089591 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.089594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090452 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.090453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.090542 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.090558 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.090638 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.090700 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":78,"thread_start_us":72,"threads_started":1}
I20260929 06:31:46.091094 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41461 (local address 127.0.0.1:56230)
0929 06:31:46.086703 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.086715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.086772 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.087122 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.087123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.087126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.087161 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.087163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088499 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.088500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.089888 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.089890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090827 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.090828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.090896 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.090900 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.090936 (+ 36us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.090990 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":90,"thread_start_us":79,"threads_started":1}
I20260929 06:31:46.091120 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40143 (local address 127.0.0.1:54474)
0929 06:31:46.086763 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.086773 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.086818 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.087126 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.087127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.087129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.087179 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.087181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088571 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.088572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.089886 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.089889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090822 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.090823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.090897 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.090902 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.090963 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.091012 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":11,"thread_start_us":46,"threads_started":1}
I20260929 06:31:46.091383 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.13.15.131:40143)
0929 06:31:46.086896 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.086901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.086911 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:31:46.086950 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.086952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.086968 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.087044 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.087268 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088453 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.089991 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090672 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.090735 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.090788 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.091235 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.091286 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260929 06:31:46.091668 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.13.15.129:41461)
0929 06:31:46.086932 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.086936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.086946 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:31:46.087003 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.087005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.087022 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.087231 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.087315 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088393 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.089980 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090725 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.091428 (+ 703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.091513 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.091546 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.091592 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":76,"threads_started":1}
I20260929 06:31:46.092095 27539 tablet_service.cc:1467] Processing CreateTablet for tablet b4608b8eff6946b6abcd29a287c47d3e (DEFAULT_TABLE table=test-table [id=03033ec0df2348b2add4a3ad0478426a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.092523 27539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4608b8eff6946b6abcd29a287c47d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.092432 27271 tablet_service.cc:1467] Processing CreateTablet for tablet b4608b8eff6946b6abcd29a287c47d3e (DEFAULT_TABLE table=test-table [id=03033ec0df2348b2add4a3ad0478426a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.092801 27271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4608b8eff6946b6abcd29a287c47d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.091574 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.086449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.13.15.130:43671)
0929 06:31:46.086663 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.086698 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.086709 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:31:46.086750 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.086752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.086767 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.087061 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.087139 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.088329 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.089688 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.090366 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.090440 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.090498 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.090674 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.090726 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260929 06:31:46.095492 27619 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Bootstrap starting.
I20260929 06:31:46.095963 27620 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Bootstrap starting.
I20260929 06:31:46.096271 27619 tablet_bootstrap.cc:654] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.096189 27405 tablet_service.cc:1467] Processing CreateTablet for tablet b4608b8eff6946b6abcd29a287c47d3e (DEFAULT_TABLE table=test-table [id=03033ec0df2348b2add4a3ad0478426a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.096626 27405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4608b8eff6946b6abcd29a287c47d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.096735 27620 tablet_bootstrap.cc:654] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.097000 27619 log.cc:824] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.097398 27620 log.cc:824] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.097888 27619 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: No bootstrap required, opened a new log
I20260929 06:31:46.097962 27619 ts_tablet_manager.cc:1397] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 06:31:46.098860 27620 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: No bootstrap required, opened a new log
I20260929 06:31:46.098932 27620 ts_tablet_manager.cc:1397] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 06:31:46.099938 27623 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Bootstrap starting.
I20260929 06:31:46.099886 27619 raft_consensus.cc:348] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.100086 27619 raft_consensus.cc:374] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.100128 27619 raft_consensus.cc:729] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74546824567a43fca8dfda381c856ef4, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.100255 27619 consensus_queue.cc:260] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.100529 27619 ts_tablet_manager.cc:1428] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:31:46.100620 27623 tablet_bootstrap.cc:654] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.100662 27605 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
I20260929 06:31:46.100970 27620 raft_consensus.cc:348] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.101166 27620 raft_consensus.cc:374] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.101184 27623 log.cc:824] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.101210 27620 raft_consensus.cc:729] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03942114d8bc446f85813895870f4041, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.101325 27620 consensus_queue.cc:260] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.101626 27620 ts_tablet_manager.cc:1428] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:31:46.101680 27337 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
I20260929 06:31:46.101943 27623 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: No bootstrap required, opened a new log
I20260929 06:31:46.102185 27623 ts_tablet_manager.cc:1397] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:31:46.103590 27623 raft_consensus.cc:348] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.103705 27623 raft_consensus.cc:374] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.103725 27623 raft_consensus.cc:729] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d907ca2ed68447a949206ecfa95f974, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.103844 27623 consensus_queue.cc:260] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.104130 27623 ts_tablet_manager.cc:1428] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:31:46.104151 27471 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
I20260929 06:31:46.115600 27626 raft_consensus.cc:482] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:46.115694 27626 raft_consensus.cc:504] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.115983 27626 leader_election.cc:288] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.119177 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.116144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43671 (local address 127.13.15.129:36241)
0929 06:31:46.116200 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.116212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.116261 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.116495 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.116496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.116498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.116574 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.116576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.117516 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.117517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.118317 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.118319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.118912 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.118994 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.118998 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.119055 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.119086 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 06:31:46.119184 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.116320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:36241 (local address 127.13.15.130:43671)
0929 06:31:46.116374 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.116376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.116384 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:31:46.116412 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.116412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.116422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.116477 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.116656 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.117435 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118402 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.118832 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118897 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.118904 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.119070 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.119119 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:31:46.119252 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.116263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:36787 (local address 127.13.15.131:40143)
0929 06:31:46.116337 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.116339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.116441 (+ 102us) server_negotiation.cc:407] Connection header received
0929 06:31:46.116496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.116497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.116524 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.116576 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.116744 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.117436 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118513 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.118893 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118965 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.118976 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.119139 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.119180 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:31:46.119277 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.116144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40143 (local address 127.13.15.129:36787)
0929 06:31:46.116368 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.116378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.116429 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.116595 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.116596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.116598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.116668 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.116670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.117550 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.117551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.118430 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.118432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.118976 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.118977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.119061 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.119065 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.119121 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.119157 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":50,"thread_start_us":124,"threads_started":1}
I20260929 06:31:46.119521 27425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4608b8eff6946b6abcd29a287c47d3e" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d907ca2ed68447a949206ecfa95f974" is_pre_election: true
I20260929 06:31:46.119710 27425 raft_consensus.cc:2454] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 0.
I20260929 06:31:46.119932 27224 leader_election.cc:302] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03942114d8bc446f85813895870f4041, 0d907ca2ed68447a949206ecfa95f974; no voters:
I20260929 06:31:46.119967 27559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4608b8eff6946b6abcd29a287c47d3e" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74546824567a43fca8dfda381c856ef4" is_pre_election: true
I20260929 06:31:46.120056 27626 raft_consensus.cc:2781] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:31:46.120105 27626 raft_consensus.cc:482] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:31:46.120141 27626 raft_consensus.cc:3037] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.120138 27559 raft_consensus.cc:2454] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 0.
I20260929 06:31:46.120844 27626 raft_consensus.cc:504] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.120992 27626 leader_election.cc:288] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 election: Requested vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.121132 27425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4608b8eff6946b6abcd29a287c47d3e" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d907ca2ed68447a949206ecfa95f974"
I20260929 06:31:46.121145 27559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4608b8eff6946b6abcd29a287c47d3e" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74546824567a43fca8dfda381c856ef4"
I20260929 06:31:46.121208 27425 raft_consensus.cc:3037] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.121248 27559 raft_consensus.cc:3037] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.121896 27425 raft_consensus.cc:2454] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 1.
I20260929 06:31:46.122026 27559 raft_consensus.cc:2454] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 1.
I20260929 06:31:46.122109 27224 leader_election.cc:302] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03942114d8bc446f85813895870f4041, 0d907ca2ed68447a949206ecfa95f974; no voters:
I20260929 06:31:46.122207 27626 raft_consensus.cc:2781] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:46.122402 27626 raft_consensus.cc:686] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 LEADER]: Becoming Leader. State: Replica: 03942114d8bc446f85813895870f4041, State: Running, Role: LEADER
I20260929 06:31:46.122522 27626 consensus_queue.cc:237] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.123308 27146 catalog_manager.cc:5495] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03942114d8bc446f85813895870f4041 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "03942114d8bc446f85813895870f4041" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } health_report { overall_health: UNKNOWN } } }
I20260929 06:31:46.136072 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.133890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.13.15.129:41461)
0929 06:31:46.133960 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.133962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.133972 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:31:46.134002 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.134003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.134017 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.134074 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:31:46.134232 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.134683 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.135409 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.135580 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.135626 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.135763 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:31:46.135850 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:31:46.135909 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.135938 (+ 29us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.135953 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:31:46.137919 27559 raft_consensus.cc:1261] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Refusing update from remote peer 03942114d8bc446f85813895870f4041: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:46.137986 27425 raft_consensus.cc:1261] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Refusing update from remote peer 03942114d8bc446f85813895870f4041: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:46.138268 27626 consensus_queue.cc:1035] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:46.138412 27629 consensus_queue.cc:1035] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:46.140515 27632 mvcc.cc:205] Tried to move back new op lower bound from 7334557721137807360 to 7334557721078603776. Current Snapshot: MvccSnapshot[applied={T|T < 7334557721137807360}]
I20260929 06:31:46.140897 27633 mvcc.cc:205] Tried to move back new op lower bound from 7334557721137807360 to 7334557721078603776. Current Snapshot: MvccSnapshot[applied={T|T < 7334557721137807360}]
I20260929 06:31:46.142747 27146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:46.142844 27146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:46.143602 27146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Sending DeleteTablet for 3 replicas of tablet b4608b8eff6946b6abcd29a287c47d3e
I20260929 06:31:46.143932 27405 tablet_service.cc:1514] Processing DeleteTablet for tablet b4608b8eff6946b6abcd29a287c47d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:53850
I20260929 06:31:46.143985 27271 tablet_service.cc:1514] Processing DeleteTablet for tablet b4608b8eff6946b6abcd29a287c47d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:56230
I20260929 06:31:46.144168 27642 tablet_replica.cc:330] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: stopping tablet replica
I20260929 06:31:46.144178 27539 tablet_service.cc:1514] Processing DeleteTablet for tablet b4608b8eff6946b6abcd29a287c47d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:54474
I20260929 06:31:46.144277 27642 raft_consensus.cc:2229] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:46.144402 27642 raft_consensus.cc:2258] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:46.144411 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27208
I20260929 06:31:46.145951 27642 ts_tablet_manager.cc:1905] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:46.146642 27644 tablet_replica.cc:330] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: stopping tablet replica
I20260929 06:31:46.146755 27644 raft_consensus.cc:2229] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:46.146871 27644 raft_consensus.cc:2258] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:46.147281 27644 ts_tablet_manager.cc:1905] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:46.148146 27642 ts_tablet_manager.cc:1918] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:46.148226 27642 log.cc:1196] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/b4608b8eff6946b6abcd29a287c47d3e
I20260929 06:31:46.148515 27642 ts_tablet_manager.cc:1939] T b4608b8eff6946b6abcd29a287c47d3e P 0d907ca2ed68447a949206ecfa95f974: Deleting consensus metadata
I20260929 06:31:46.148818 27644 ts_tablet_manager.cc:1918] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:46.148895 27644 log.cc:1196] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/b4608b8eff6946b6abcd29a287c47d3e
I20260929 06:31:46.148952 27130 catalog_manager.cc:4841] TS 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): tablet b4608b8eff6946b6abcd29a287c47d3e (table test-table [id=03033ec0df2348b2add4a3ad0478426a]) successfully deleted
I20260929 06:31:46.149150 27644 ts_tablet_manager.cc:1939] T b4608b8eff6946b6abcd29a287c47d3e P 74546824567a43fca8dfda381c856ef4: Deleting consensus metadata
I20260929 06:31:46.149524 27133 catalog_manager.cc:4841] TS 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): tablet b4608b8eff6946b6abcd29a287c47d3e (table test-table [id=03033ec0df2348b2add4a3ad0478426a]) successfully deleted
W20260929 06:31:46.150547 27133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:41461 (error 108)
I20260929 06:31:46.150668 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27342
I20260929 06:31:46.155500 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27476
I20260929 06:31:46.155681 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.155477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41461 (local address 127.0.0.1:56240)
0929 06:31:46.155584 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.155643 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 06:31:46.156077 27133 catalog_manager.cc:4584] TS 03942114d8bc446f85813895870f4041 (127.13.15.129:41461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4608b8eff6946b6abcd29a287c47d3e: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41461: connect: Connection refused (error 111)
I20260929 06:31:46.160092 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27114
I20260929 06:31:46.165004 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--webserver_interface=127.13.15.190
--webserver_port=37403
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:46.244534 27646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:46.244719 27646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:46.244741 27646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:46.245806 27646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:46.245841 27646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:46.245854 27646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:46.245867 27646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:46.245879 27646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:46.247262 27646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=37403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:46.247440 27646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:46.247599 27646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:46.255621 27653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.255678 27652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.255645 27655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.255861 27646 server_base.cc:1034] running on GCE node
I20260929 06:31:46.256121 27646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:46.256421 27646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:46.257563 27646 hybrid_clock.cc:648] HybridClock initialized: now 1790663506257556 us; error 47 us; skew 500 ppm
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:46.260452 27646 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:46.260896 27646 webserver.cc:460] Webserver started at http://127.13.15.190:37403/ using document root <none> and password file <none>
I20260929 06:31:46.261137 27646 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:46.261190 27646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:46.262544 27646 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:46.263155 27662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.263336 27646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:31:46.263434 27646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "369bc7159509454fbd4c92767e8c9606"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "1b3bd0dc60ad38f45211d9015fe022d5"
server_key_iv: "b848612d0fefb84a0798a415ff4b3ed7"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.263803 27646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:46.287586 27646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:46.289839 27646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:46.289994 27646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:46.293843 27646 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44701
I20260929 06:31:46.293896 27714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44701 every 8 connection(s)
I20260929 06:31:46.294313 27646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:46.295769 27715 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:31:46.296418 27715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap starting.
I20260929 06:31:46.299598 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27646
I20260929 06:31:46.299937 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:41461
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=35475
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:46.304921 27715 log.cc:824] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.306921 27715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:31:46.307216 27715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap complete.
I20260929 06:31:46.309450 27715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:46.309664 27715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.309826 27715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:46.309891 27715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:46.309943 27715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:46.309991 27715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:31:46.311004 27715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:46.311087 27715 leader_election.cc:302] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369bc7159509454fbd4c92767e8c9606; no voters:
I20260929 06:31:46.311266 27715 leader_election.cc:288] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 06:31:46.311300 27719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 06:31:46.311467 27719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 LEADER]: Becoming Leader. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Running, Role: LEADER
I20260929 06:31:46.311529 27715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:46.311623 27719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:46.311894 27719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369bc7159509454fbd4c92767e8c9606. Latest consensus state: current_term: 2 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:46.311898 27721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:46.312047 27719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:46.312251 27721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:46.312623 27726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:46.313519 27726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:31:46.313629 27726 catalog_manager.cc:1259] Loaded cluster ID: 8ba1c81da48144d3aec2649aa2a7ea1d
I20260929 06:31:46.313663 27726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:31:46.313894 27726 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 06:31:46.316653 27726 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 06:31:46.318534 27726 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:31:46.318735 27726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Loaded TSK: 0
I20260929 06:31:46.318969 27726 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:31:46.410486 27717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:46.410744 27717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:46.410778 27717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:46.412380 27717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:46.412438 27717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:46.412477 27717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:31:46.414462 27717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:41461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=35475
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:46.414701 27717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:46.414945 27717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:46.420147 27745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.420176 27743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.420418 27717 server_base.cc:1034] running on GCE node
W20260929 06:31:46.420533 27742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.420717 27717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:46.421006 27717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:46.422191 27717 hybrid_clock.cc:648] HybridClock initialized: now 1790663506422172 us; error 34 us; skew 500 ppm
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:46.424716 27717 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:31:46.425237 27717 webserver.cc:460] Webserver started at http://127.13.15.129:35475/ using document root <none> and password file <none>
I20260929 06:31:46.425494 27717 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:46.425577 27717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:46.427299 27717 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:46.428082 27752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.428282 27717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:46.428383 27717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "03942114d8bc446f85813895870f4041"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "45b63f32120b60c3f5b85e13d388be28"
server_key_iv: "6695174ed12b38af0827d5c559318391"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.428776 27717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:46.442381 27717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:46.444519 27717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:46.444640 27717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:46.444913 27717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:46.445396 27759 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 06:31:46.446780 27717 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 06:31:46.446827 27717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 06:31:46.446858 27717 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 06:31:46.447481 27717 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 06:31:46.447525 27717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 06:31:46.447563 27759 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Bootstrap starting.
I20260929 06:31:46.454433 27717 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:41461
I20260929 06:31:46.454972 27717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:31:46.455118 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27717
I20260929 06:31:46.455415 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:43671
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=36509
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:46.457652 27866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:41461 every 8 connection(s)
I20260929 06:31:46.458112 27759 log.cc:824] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.461297 27759 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:31:46.461612 27759 tablet_bootstrap.cc:492] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Bootstrap complete.
I20260929 06:31:46.462874 27759 ts_tablet_manager.cc:1397] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
I20260929 06:31:46.464542 27759 raft_consensus.cc:348] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.465011 27759 raft_consensus.cc:729] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03942114d8bc446f85813895870f4041, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.465169 27759 consensus_queue.cc:260] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } } peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } }
I20260929 06:31:46.465545 27759 ts_tablet_manager.cc:1428] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:46.471101 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.458250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:50877)
0929 06:31:46.458557 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.458603 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.459228 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.459950 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.459954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.460341 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:46.460609 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.460615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.462310 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.462311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.463551 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.463554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.464443 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.464444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.464571 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.465208 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:46.465217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:46.467115 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:46.469345 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.469351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.469353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.469631 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.469901 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.469902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.469903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.470036 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.470492 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:46.470499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:46.470640 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.470755 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.470916 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":83,"threads_started":1}
I20260929 06:31:46.471128 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.458355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:50877 (local address 127.13.15.190:44701)
0929 06:31:46.459440 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.459451 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.459488 (+ 37us) server_negotiation.cc:407] Connection header received
0929 06:31:46.459612 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.459618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.459671 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.459797 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.460743 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.462201 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.463658 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.464254 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.464352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.467228 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.467232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.467232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.467243 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.469257 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.469724 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.469727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.469727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.469826 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.470113 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.470114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.470115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.470410 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.470489 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.470819 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.470944 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":78,"threads_started":1}
I20260929 06:31:46.473093 27867 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:46.473208 27867 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:46.473409 27867 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:46.474200 27679 ts_manager.cc:194] Registered new tserver with Master: 03942114d8bc446f85813895870f4041 (127.13.15.129:41461)
I20260929 06:31:46.475391 27679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:50877
I20260929 06:31:46.476352 27867 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
I20260929 06:31:46.480585 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.474834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41461 (local address 127.0.0.1:56256)
0929 06:31:46.475914 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.475946 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.476010 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.476358 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.476361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.476362 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.476434 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.476436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.477912 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.477913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.479364 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.479367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.480268 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.480269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.480371 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.480384 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.480459 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.480505 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"thread_start_us":112,"threads_started":1}
I20260929 06:31:46.480593 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.475674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.13.15.129:41461)
0929 06:31:46.475812 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.475816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.476078 (+ 262us) server_negotiation.cc:407] Connection header received
0929 06:31:46.476109 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.476111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.476237 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.476302 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.476686 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.477819 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.479460 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.480159 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.480237 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.480294 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.480486 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.480532 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260929 06:31:46.481315 27791 tablet_service.cc:1514] Processing DeleteTablet for tablet b4608b8eff6946b6abcd29a287c47d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:56256
I20260929 06:31:46.481894 27876 tablet_replica.cc:330] stopping tablet replica
I20260929 06:31:46.481998 27876 raft_consensus.cc:2229] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:46.482055 27876 raft_consensus.cc:2258] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:46.482354 27876 ts_tablet_manager.cc:1905] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:46.484769 27876 ts_tablet_manager.cc:1918] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:46.484872 27876 log.cc:1196] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/b4608b8eff6946b6abcd29a287c47d3e
I20260929 06:31:46.485211 27876 ts_tablet_manager.cc:1939] T b4608b8eff6946b6abcd29a287c47d3e P 03942114d8bc446f85813895870f4041: Deleting consensus metadata
I20260929 06:31:46.485761 27666 catalog_manager.cc:4841] TS 03942114d8bc446f85813895870f4041 (127.13.15.129:41461): tablet b4608b8eff6946b6abcd29a287c47d3e (table test-table [id=03033ec0df2348b2add4a3ad0478426a]) successfully deleted
W20260929 06:31:46.542958 27869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:46.543183 27869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:46.543233 27869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:46.544371 27869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:46.544436 27869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:46.544478 27869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:31:46.545949 27869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=36509
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:46.546239 27869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:46.546464 27869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:31:46.551549 27869 server_base.cc:1034] running on GCE node
W20260929 06:31:46.551496 27881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.551517 27884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.551496 27882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.551959 27869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:46.552146 27869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:46.553298 27869 hybrid_clock.cc:648] HybridClock initialized: now 1790663506553287 us; error 24 us; skew 500 ppm
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:46.555630 27869 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:31:46.556034 27869 webserver.cc:460] Webserver started at http://127.13.15.130:36509/ using document root <none> and password file <none>
I20260929 06:31:46.556267 27869 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:46.556356 27869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:46.557619 27869 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:46.558200 27891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.558372 27869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:31:46.558466 27869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "0d907ca2ed68447a949206ecfa95f974"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "e2af4e1df37b9bb4c3560f08e28f5976"
server_key_iv: "c093e8c8d30470788a4c3c70924d2609"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.558792 27869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:46.578738 27869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:46.580483 27869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:46.580639 27869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:46.580865 27869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:46.581161 27869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:46.581215 27869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.581254 27869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:46.581285 27869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.587028 27869 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43671
I20260929 06:31:46.587072 28004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43671 every 8 connection(s)
I20260929 06:31:46.587406 27869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:31:46.591492 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 27869
I20260929 06:31:46.591789 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:40143
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=44609
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:46.596480 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.588091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:41371 (local address 127.13.15.190:44701)
0929 06:31:46.588148 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.588150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.588336 (+ 186us) server_negotiation.cc:407] Connection header received
0929 06:31:46.588816 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.588817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.588827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.588886 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.589746 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.590296 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.591216 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.591576 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.591651 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.593823 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.593824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.593825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.593833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.595174 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.595565 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.595566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.595567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.595623 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.595870 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.595871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.595871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.595976 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.596042 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.596312 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.596392 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:31:46.596642 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.588072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:41371)
0929 06:31:46.588212 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.588251 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.588738 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.588947 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.588949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.589492 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:46.589670 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.589674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.590377 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.590378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.591147 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.591149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.591665 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.591666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.591776 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.592256 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:46.592266 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:46.593729 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:46.595256 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.595262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.595264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.595489 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.595688 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.595688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.595689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.595793 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.596046 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:46.596052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:46.596177 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.596277 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.596428 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260929 06:31:46.597133 28005 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:46.597204 28005 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:46.597393 28005 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:46.597779 27679 ts_manager.cc:194] Registered new tserver with Master: 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671)
I20260929 06:31:46.598367 27679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:41371
W20260929 06:31:46.672438 28008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:46.672641 28008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:46.672663 28008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:46.673717 28008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:46.673750 28008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:46.673775 28008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:31:46.675312 28008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:40143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=44609
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:46.675484 28008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:46.675643 28008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:46.680576 28014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.680584 28016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:46.680588 28013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:46.680688 28008 server_base.cc:1034] running on GCE node
I20260929 06:31:46.681090 28008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:46.681320 28008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:46.682476 28008 hybrid_clock.cc:648] HybridClock initialized: now 1790663506682469 us; error 34 us; skew 500 ppm
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:46.684892 28008 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:31:46.685381 28008 webserver.cc:460] Webserver started at http://127.13.15.131:44609/ using document root <none> and password file <none>
I20260929 06:31:46.685597 28008 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:46.685676 28008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:46.687065 28008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:46.687620 28023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.687773 28008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:31:46.687882 28008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "74546824567a43fca8dfda381c856ef4"
format_stamp: "Formatted at 2026-09-29 06:31:46 on dist-test-slave-sm0p"
server_key: "647c7881728aea89f7cf334df9ba7f81"
server_key_iv: "884868f257ede6273352f3534e432617"
server_key_version: "encryptionkey@0"
I20260929 06:31:46.688205 28008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:46.699797 28008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:46.701982 28008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:46.702167 28008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:46.702396 28008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:46.702679 28008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:46.702730 28008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.702769 28008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:46.702795 28008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.708495 28008 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40143
I20260929 06:31:46.708546 28136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40143 every 8 connection(s)
I20260929 06:31:46.708860 28008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:31:46 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663506, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:31:46.716645 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28008
I20260929 06:31:46.717602 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.709544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:60205 (local address 127.13.15.190:44701)
0929 06:31:46.709605 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.709607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.709788 (+ 181us) server_negotiation.cc:407] Connection header received
0929 06:31:46.710266 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.710267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.710276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.710337 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.711176 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.711637 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.712736 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.712961 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.713013 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.714976 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.714977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.714977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.714984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.716138 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.716491 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.716493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.716493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.716558 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.716927 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.716928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.716928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.717078 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.717158 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.717432 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.717511 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:31:46.717779 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.709520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:60205)
0929 06:31:46.709661 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.709701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.710186 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.710392 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.710395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.710932 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:46.711105 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.711109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.711754 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.711755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.712658 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.712660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.713027 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.713028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.713124 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.713597 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:46.713605 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:46.714893 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:46.716206 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.716211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.716213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.716417 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.716651 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:46.716652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:46.716653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:46.716827 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:46.717152 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:46.717158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:46.717294 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.717411 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.717564 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260929 06:31:46.718245 28137 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44701
I20260929 06:31:46.718330 28137 heartbeater.cc:464] Registering TS with master...
I20260929 06:31:46.718515 28137 heartbeater.cc:511] Master 127.13.15.190:44701 requested a full tablet report, sending...
I20260929 06:31:46.718884 27679 ts_manager.cc:194] Registered new tserver with Master: 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.719247 27679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:60205
I20260929 06:31:46.723888 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.717076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.13.15.190:44701)
0929 06:31:46.717200 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.717202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.717997 (+ 795us) server_negotiation.cc:407] Connection header received
0929 06:31:46.718048 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.718049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.718061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.718125 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:46.718799 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.719395 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.720127 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.720347 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.720420 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.721541 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:46.721543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:46.721543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:46.721550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:46.722778 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.723119 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.723120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.723120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.723175 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:46.723382 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:46.723383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:46.723383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:46.723488 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:46.723678 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.723715 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.723786 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260929 06:31:46.724543 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:31:46.726985 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.724833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.13.15.190:44701)
0929 06:31:46.724898 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.724900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.725028 (+ 128us) server_negotiation.cc:407] Connection header received
0929 06:31:46.725091 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.725091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.725117 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.725171 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:31:46.725253 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.725675 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.726406 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.726583 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.726633 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.726750 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:31:46.726830 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:31:46.726895 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.726925 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.726942 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 06:31:46.727241 27679 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:31:46.728693 27679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:31:46.730868 27791 tablet_service.cc:1467] Processing CreateTablet for tablet 977077f357224202a960bd6395ceb647 (DEFAULT_TABLE table=test-table [id=1b686174f0334c408a6c4446c7f035ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.731077 27791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977077f357224202a960bd6395ceb647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.732631 27759 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Bootstrap starting.
I20260929 06:31:46.732959 27759 tablet_bootstrap.cc:654] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.734017 27759 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: No bootstrap required, opened a new log
I20260929 06:31:46.734066 27759 ts_tablet_manager.cc:1397] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:46.734186 27759 raft_consensus.cc:348] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.734252 27759 raft_consensus.cc:374] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.734292 27759 raft_consensus.cc:729] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03942114d8bc446f85813895870f4041, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.734339 27759 consensus_queue.cc:260] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.734445 27759 ts_tablet_manager.cc:1428] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:46.734591 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.730453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43671 (local address 127.0.0.1:53856)
0929 06:31:46.730506 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.730516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.730553 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.730920 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.730921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.730922 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.730950 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.730951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.732302 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.732302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.733374 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.733376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.734357 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.734358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.734414 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.734418 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.734475 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.734516 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":45}
I20260929 06:31:46.734618 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.730560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.13.15.130:43671)
0929 06:31:46.730744 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.730748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.730760 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:31:46.730800 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.730802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.730817 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.730921 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.731051 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.732191 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.733479 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.734267 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.734353 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.734412 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.734489 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.734542 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260929 06:31:46.734824 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.730453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40143 (local address 127.0.0.1:54480)
0929 06:31:46.730648 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.730657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.730693 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.731262 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.731263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.731265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.731304 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.731305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.732496 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.732497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.733577 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.733580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.734454 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.734455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.734524 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.734541 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.734575 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.734609 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":22,"thread_start_us":79,"threads_started":1}
I20260929 06:31:46.735121 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.730551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.13.15.131:40143)
0929 06:31:46.731050 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.731054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.731063 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:31:46.731097 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.731099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.731114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.731190 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.731388 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.732393 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.733643 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.734340 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.734907 (+ 567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.734967 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.734991 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.735029 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":95,"threads_started":1}
I20260929 06:31:46.735301 27939 tablet_service.cc:1467] Processing CreateTablet for tablet 977077f357224202a960bd6395ceb647 (DEFAULT_TABLE table=test-table [id=1b686174f0334c408a6c4446c7f035ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.735647 27939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977077f357224202a960bd6395ceb647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.735795 28071 tablet_service.cc:1467] Processing CreateTablet for tablet 977077f357224202a960bd6395ceb647 (DEFAULT_TABLE table=test-table [id=1b686174f0334c408a6c4446c7f035ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:31:46.736104 28071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977077f357224202a960bd6395ceb647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:31:46.738313 28147 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Bootstrap starting.
I20260929 06:31:46.738593 28148 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Bootstrap starting.
I20260929 06:31:46.739020 28147 tablet_bootstrap.cc:654] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.739272 28148 tablet_bootstrap.cc:654] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Neither blocks nor log segments found. Creating new log.
I20260929 06:31:46.739651 28147 log.cc:824] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.739814 28148 log.cc:824] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:46.740381 28147 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: No bootstrap required, opened a new log
I20260929 06:31:46.740460 28147 ts_tablet_manager.cc:1397] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:46.740550 28148 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: No bootstrap required, opened a new log
I20260929 06:31:46.740633 28148 ts_tablet_manager.cc:1397] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:46.741804 28147 raft_consensus.cc:348] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.741963 28147 raft_consensus.cc:374] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.742004 28147 raft_consensus.cc:729] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d907ca2ed68447a949206ecfa95f974, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.741900 28148 raft_consensus.cc:348] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.742049 28148 raft_consensus.cc:374] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:31:46.742089 28148 raft_consensus.cc:729] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74546824567a43fca8dfda381c856ef4, State: Initialized, Role: FOLLOWER
I20260929 06:31:46.742158 28147 consensus_queue.cc:260] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.742228 28148 consensus_queue.cc:260] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.742475 28005 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
I20260929 06:31:46.742513 28148 ts_tablet_manager.cc:1428] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:46.742460 28147 ts_tablet_manager.cc:1428] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:31:46.742724 28137 heartbeater.cc:503] Master 127.13.15.190:44701 was elected leader, sending a full tablet report...
W20260929 06:31:46.837975 28006 tablet.cc:2369] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:31:46.905124 27873 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:46.905257 27873 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.905638 27873 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.909655 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.905987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:48469 (local address 127.13.15.130:43671)
0929 06:31:46.906036 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.906038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.906176 (+ 138us) server_negotiation.cc:407] Connection header received
0929 06:31:46.906272 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.906273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.906283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.906333 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.906507 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.907226 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.908685 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.909265 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.909352 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.909360 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.909549 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.909586 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:31:46.909655 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.906039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:40953 (local address 127.13.15.131:40143)
0929 06:31:46.906075 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.906077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.906086 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:31:46.906121 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.906122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.906147 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.906234 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:31:46.906725 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.907410 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.908685 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.909256 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.909359 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.909366 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.909549 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.909586 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:31:46.909696 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.905875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40143 (local address 127.13.15.129:40953)
0929 06:31:46.905938 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.905951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.906000 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.906252 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.906254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.906256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.906644 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.906646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.907485 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.907486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.908590 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.908592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.909367 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.909368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.909451 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.909454 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.909512 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.909584 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":3}
I20260929 06:31:46.909706 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.905877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43671 (local address 127.13.15.129:48469)
0929 06:31:46.906085 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.906099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:46.906196 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:46.906377 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:46.906378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:46.906380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:31:46.906433 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.906435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.907309 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.907310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.908590 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:46.908592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.909392 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.909393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:46.909452 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.909454 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:31:46.909512 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 06:31:46.909579 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":68,"thread_start_us":111,"threads_started":1}
I20260929 06:31:46.910140 28091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977077f357224202a960bd6395ceb647" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74546824567a43fca8dfda381c856ef4" is_pre_election: true
I20260929 06:31:46.910189 27959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977077f357224202a960bd6395ceb647" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d907ca2ed68447a949206ecfa95f974" is_pre_election: true
I20260929 06:31:46.910341 28091 raft_consensus.cc:2454] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 0.
I20260929 06:31:46.910348 27959 raft_consensus.cc:2454] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 0.
I20260929 06:31:46.910555 27753 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03942114d8bc446f85813895870f4041, 0d907ca2ed68447a949206ecfa95f974; no voters:
I20260929 06:31:46.910718 27873 raft_consensus.cc:2781] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:31:46.910778 27873 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:31:46.910802 27873 raft_consensus.cc:3037] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.911549 27873 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.911692 27873 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 election: Requested vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
I20260929 06:31:46.911866 27959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977077f357224202a960bd6395ceb647" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d907ca2ed68447a949206ecfa95f974"
I20260929 06:31:46.911885 28091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977077f357224202a960bd6395ceb647" candidate_uuid: "03942114d8bc446f85813895870f4041" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74546824567a43fca8dfda381c856ef4"
I20260929 06:31:46.911944 27959 raft_consensus.cc:3037] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.911962 28091 raft_consensus.cc:3037] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:31:46.912765 28091 raft_consensus.cc:2454] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 1.
I20260929 06:31:46.912776 27959 raft_consensus.cc:2454] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03942114d8bc446f85813895870f4041 in term 1.
I20260929 06:31:46.912976 27753 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03942114d8bc446f85813895870f4041, 0d907ca2ed68447a949206ecfa95f974; no voters:
I20260929 06:31:46.913065 27873 raft_consensus.cc:2781] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:31:46.913192 27873 raft_consensus.cc:686] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 LEADER]: Becoming Leader. State: Replica: 03942114d8bc446f85813895870f4041, State: Running, Role: LEADER
I20260929 06:31:46.913269 27873 consensus_queue.cc:237] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:46.914103 27678 catalog_manager.cc:5495] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03942114d8bc446f85813895870f4041 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "03942114d8bc446f85813895870f4041" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } health_report { overall_health: HEALTHY } } }
W20260929 06:31:46.958452 27868 tablet.cc:2369] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:31:46.959359 28138 tablet.cc:2369] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:31:46.960035 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.957588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.13.15.129:41461)
0929 06:31:46.957662 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:46.957665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:46.957671 (+ 6us) server_negotiation.cc:407] Connection header received
0929 06:31:46.957705 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:46.957706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:46.957724 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:46.957803 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:31:46.957918 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.958409 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.959211 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:46.959486 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:46.959552 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:46.959744 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:31:46.959857 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:31:46.959922 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:46.959961 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 06:31:46.959981 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:31:46.961548 28091 raft_consensus.cc:1261] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Refusing update from remote peer 03942114d8bc446f85813895870f4041: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:46.961549 27959 raft_consensus.cc:1261] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Refusing update from remote peer 03942114d8bc446f85813895870f4041: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:31:46.961912 28154 consensus_queue.cc:1035] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:46.962091 27873 consensus_queue.cc:1035] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:31:46.964715 28158 mvcc.cc:205] Tried to move back new op lower bound from 7334557724511784960 to 7334557724317540352. Current Snapshot: MvccSnapshot[applied={T|T < 7334557724511784960}]
I20260929 06:31:46.966866 27678 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:46.966943 27678 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:31:46.967702 27678 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Sending DeleteTablet for 3 replicas of tablet 977077f357224202a960bd6395ceb647
I20260929 06:31:46.968024 27939 tablet_service.cc:1514] Processing DeleteTablet for tablet 977077f357224202a960bd6395ceb647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:53856
I20260929 06:31:46.968050 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27717
I20260929 06:31:46.968204 28071 tablet_service.cc:1514] Processing DeleteTablet for tablet 977077f357224202a960bd6395ceb647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:31:46 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:54480
I20260929 06:31:46.968922 28166 tablet_replica.cc:330] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: stopping tablet replica
I20260929 06:31:46.968998 28166 raft_consensus.cc:2229] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:46.969065 28166 raft_consensus.cc:2258] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:46.969448 28166 ts_tablet_manager.cc:1905] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:46.970876 28166 ts_tablet_manager.cc:1918] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:46.970963 28166 log.cc:1196] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/977077f357224202a960bd6395ceb647
I20260929 06:31:46.971241 28166 ts_tablet_manager.cc:1939] T 977077f357224202a960bd6395ceb647 P 0d907ca2ed68447a949206ecfa95f974: Deleting consensus metadata
I20260929 06:31:46.971621 27663 catalog_manager.cc:4841] TS 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): tablet 977077f357224202a960bd6395ceb647 (table test-table [id=1b686174f0334c408a6c4446c7f035ce]) successfully deleted
W20260929 06:31:46.973845 27666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:41461 (error 108)
I20260929 06:31:46.974082 28167 tablet_replica.cc:330] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: stopping tablet replica
I20260929 06:31:46.974249 28167 raft_consensus.cc:2229] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:31:46.974388 28167 raft_consensus.cc:2258] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:31:46.974437 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:46.974312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41461 (local address 127.0.0.1:56276)
0929 06:31:46.974363 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:46.974403 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 06:31:46.974633 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27869
W20260929 06:31:46.975637 27666 catalog_manager.cc:4584] TS 03942114d8bc446f85813895870f4041 (127.13.15.129:41461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 977077f357224202a960bd6395ceb647: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41461: connect: Connection refused (error 111)
I20260929 06:31:46.976701 28167 ts_tablet_manager.cc:1905] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:31:46.978650 28167 ts_tablet_manager.cc:1918] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:31:46.978731 28167 log.cc:1196] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/977077f357224202a960bd6395ceb647
I20260929 06:31:46.979017 28167 ts_tablet_manager.cc:1939] T 977077f357224202a960bd6395ceb647 P 74546824567a43fca8dfda381c856ef4: Deleting consensus metadata
I20260929 06:31:46.979499 27666 catalog_manager.cc:4841] TS 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): tablet 977077f357224202a960bd6395ceb647 (table test-table [id=1b686174f0334c408a6c4446c7f035ce]) successfully deleted
I20260929 06:31:46.982407 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28008
I20260929 06:31:46.986975 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 27646
I20260929 06:31:46.992079 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--webserver_interface=127.13.15.190
--webserver_port=37403
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:47.069264 28169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:47.069447 28169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:47.069468 28169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:47.070595 28169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:31:47.070634 28169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:47.070647 28169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:47.070659 28169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:31:47.070672 28169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:31:47.072130 28169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=37403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:47.072368 28169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:47.072592 28169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:31:47.077478 28169 server_base.cc:1034] running on GCE node
W20260929 06:31:47.077411 28174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.077410 28175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.077389 28177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:47.077951 28169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:47.078265 28169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:47.079481 28169 hybrid_clock.cc:648] HybridClock initialized: now 1790663507079464 us; error 33 us; skew 500 ppm
Sep 29 06:31:47 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:47.081990 28169 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:31:47.082583 28169 webserver.cc:460] Webserver started at http://127.13.15.190:37403/ using document root <none> and password file <none>
I20260929 06:31:47.082859 28169 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:47.082934 28169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:47.084311 28169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:47.084996 28184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.085168 28169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:31:47.085290 28169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "369bc7159509454fbd4c92767e8c9606"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "1b3bd0dc60ad38f45211d9015fe022d5"
server_key_iv: "b848612d0fefb84a0798a415ff4b3ed7"
server_key_version: "encryptionkey@0"
I20260929 06:31:47.085649 28169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:47.100921 28169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:47.102862 28169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:47.103034 28169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:47.107479 28169 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44701
I20260929 06:31:47.107519 28236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44701 every 8 connection(s)
I20260929 06:31:47.107914 28169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:31:47.109458 28237 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:31:47.110041 28237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap starting.
I20260929 06:31:47.116971 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28169
I20260929 06:31:47.117275 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:41461
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=35475
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:31:47.118431 28237 log.cc:824] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:47.120651 28237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:31:47.120940 28237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606: Bootstrap complete.
I20260929 06:31:47.123148 28237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:47.123365 28237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Initialized, Role: FOLLOWER
I20260929 06:31:47.123530 28237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:47.123600 28237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:31:47.123649 28237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:31:47.123695 28237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 2 FOLLOWER]: Advancing to term 3
I20260929 06:31:47.124583 28237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:47.124691 28237 leader_election.cc:302] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369bc7159509454fbd4c92767e8c9606; no voters:
I20260929 06:31:47.124900 28237 leader_election.cc:288] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 06:31:47.124950 28241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 06:31:47.125072 28241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [term 3 LEADER]: Becoming Leader. State: Replica: 369bc7159509454fbd4c92767e8c9606, State: Running, Role: LEADER
I20260929 06:31:47.125181 28237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:31:47.125159 28241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } }
I20260929 06:31:47.125438 28242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:47.125536 28242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:47.125774 28243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369bc7159509454fbd4c92767e8c9606. Latest consensus state: current_term: 3 leader_uuid: "369bc7159509454fbd4c92767e8c9606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369bc7159509454fbd4c92767e8c9606" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44701 } } }
I20260929 06:31:47.125900 28243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369bc7159509454fbd4c92767e8c9606 [sys.catalog]: This master's current role is: LEADER
I20260929 06:31:47.126029 28246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:31:47.126905 28246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:31:47.127040 28246 catalog_manager.cc:1259] Loaded cluster ID: 8ba1c81da48144d3aec2649aa2a7ea1d
I20260929 06:31:47.127086 28246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 06:31:47.127321 28246 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3889ed59ad google::LogMessage::Fail() at ??:0
@ 0x7f3889ed888a google::LogMessage::SendToLog() at ??:0
@ 0x7f3889ed54de google::LogMessage::Flush() at ??:0
@ 0x7f3889ed6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3891df60f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3891dff8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f388bda30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f388bd98954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f388cbfc6db start_thread at ??:0
@ 0x7f38889d361f clone at ??:0
W20260929 06:31:47.210214 28239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:47.210417 28239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:47.210449 28239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:47.211522 28239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:47.211585 28239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:47.211623 28239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:31:47.213017 28239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:41461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=35475
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:47.213269 28239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:47.213487 28239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:47.218266 28273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:47.218331 28239 server_base.cc:1034] running on GCE node
W20260929 06:31:47.218254 28276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.218585 28274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:47.218808 28239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:47.219050 28239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:47.220203 28239 hybrid_clock.cc:648] HybridClock initialized: now 1790663507220189 us; error 29 us; skew 500 ppm
Sep 29 06:31:47 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:47.222555 28239 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:31:47.223011 28239 webserver.cc:460] Webserver started at http://127.13.15.129:35475/ using document root <none> and password file <none>
I20260929 06:31:47.223249 28239 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:47.223294 28239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:47.225404 28239 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 06:31:47.226118 28283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.226294 28239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:47.226385 28239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "03942114d8bc446f85813895870f4041"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "45b63f32120b60c3f5b85e13d388be28"
server_key_iv: "6695174ed12b38af0827d5c559318391"
server_key_version: "encryptionkey@0"
I20260929 06:31:47.226723 28239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:47.239219 28239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:47.241353 28239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:47.241503 28239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:47.241708 28239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:47.242224 28290 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 06:31:47.244180 28239 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 06:31:47.244263 28239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 06:31:47.244308 28239 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 06:31:47.244930 28239 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 06:31:47.244987 28239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 06:31:47.245048 28290 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Bootstrap starting.
I20260929 06:31:47.251914 28239 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:41461
I20260929 06:31:47.252010 28397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:41461 every 8 connection(s)
I20260929 06:31:47.252301 28239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:31:47.255810 28290 log.cc:824] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Log is configured to *not* fsync() on all Append() calls
I20260929 06:31:47.258548 28290 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:31:47.258921 28290 tablet_bootstrap.cc:492] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Bootstrap complete.
I20260929 06:31:47.260138 28290 ts_tablet_manager.cc:1397] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260929 06:31:47.261502 28290 raft_consensus.cc:348] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:47.261837 28290 raft_consensus.cc:729] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03942114d8bc446f85813895870f4041, State: Initialized, Role: FOLLOWER
I20260929 06:31:47.261969 28290 consensus_queue.cc:260] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:47.262236 28290 ts_tablet_manager.cc:1428] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260929 06:31:47.262428 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28239
I20260929 06:31:47.262729 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:43671
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=36509
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:47.321323 28400 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:31:47.252976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:39865)
0929 06:31:47.253148 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.253187 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:47.253594 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:47.321150 (+ 67556us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
W20260929 06:31:47.321600 28398 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44701 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:44701: Client connection negotiation failed: client connection to 127.13.15.190:44701: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 06:31:47.321959 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.321852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:46953)
0929 06:31:47.321902 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.321939 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 06:31:47.322384 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.322292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:39403)
0929 06:31:47.322337 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.322361 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 06:31:47.322515 28398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:31:47.345713 28403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:47.345890 28403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:47.345914 28403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:47.347096 28403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:47.347141 28403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:47.347200 28403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:31:47.348636 28403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=36509
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:47.348870 28403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:47.349093 28403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:47.353979 28408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.354038 28411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.354100 28409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:47.354435 28403 server_base.cc:1034] running on GCE node
I20260929 06:31:47.354635 28403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:47.354813 28403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:47.355942 28403 hybrid_clock.cc:648] HybridClock initialized: now 1790663507355934 us; error 31 us; skew 500 ppm
Sep 29 06:31:47 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:47.358390 28403 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:31:47.358793 28403 webserver.cc:460] Webserver started at http://127.13.15.130:36509/ using document root <none> and password file <none>
I20260929 06:31:47.359014 28403 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:47.359081 28403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:47.360278 28403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:31:47.360841 28418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.360986 28403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:31:47.361074 28403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "0d907ca2ed68447a949206ecfa95f974"
format_stamp: "Formatted at 2026-09-29 06:31:45 on dist-test-slave-sm0p"
server_key: "e2af4e1df37b9bb4c3560f08e28f5976"
server_key_iv: "c093e8c8d30470788a4c3c70924d2609"
server_key_version: "encryptionkey@0"
I20260929 06:31:47.361374 28403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:47.382637 28403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:47.384315 28403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:47.384464 28403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:47.384667 28403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:47.384980 28403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:47.385035 28403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.385075 28403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:47.385103 28403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.390941 28403 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:43671
I20260929 06:31:47.390987 28531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:43671 every 8 connection(s)
I20260929 06:31:47.391292 28403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:31:47.392280 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.391938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:58621)
0929 06:31:47.392083 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.392151 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
W20260929 06:31:47.392649 28532 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44701 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:44701: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
I20260929 06:31:47.392917 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.392824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:59481)
0929 06:31:47.392870 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.392894 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 06:31:47.393174 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.393093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:54565)
0929 06:31:47.393135 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.393156 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:31:47.393281 28532 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:31:47.397971 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28403
I20260929 06:31:47.398284 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:40143
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:44701
--webserver_port=44609
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:31:47.476715 28535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:31:47.476898 28535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:31:47.476920 28535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:31:47.477984 28535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:31:47.478018 28535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:31:47.478044 28535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:31:47.479512 28535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:40143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=44609
--tserver_master_addrs=127.13.15.190:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:31:47.479676 28535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:31:47.479847 28535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:31:47.484565 28541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.484560 28543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:31:47.484557 28540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:31:47.484810 28535 server_base.cc:1034] running on GCE node
I20260929 06:31:47.485025 28535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:31:47.485250 28535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:31:47.486402 28535 hybrid_clock.cc:648] HybridClock initialized: now 1790663507486397 us; error 39 us; skew 500 ppm
Sep 29 06:31:47 dist-test-slave-sm0p krb5kdc[27101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:31:47.488690 28535 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:31:47.489149 28535 webserver.cc:460] Webserver started at http://127.13.15.131:44609/ using document root <none> and password file <none>
I20260929 06:31:47.489365 28535 fs_manager.cc:359] Metadata directory not provided
I20260929 06:31:47.489430 28535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:31:47.490712 28535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:31:47.491225 28550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.491390 28535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.491482 28535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "74546824567a43fca8dfda381c856ef4"
format_stamp: "Formatted at 2026-09-29 06:31:46 on dist-test-slave-sm0p"
server_key: "647c7881728aea89f7cf334df9ba7f81"
server_key_iv: "884868f257ede6273352f3534e432617"
server_key_version: "encryptionkey@0"
I20260929 06:31:47.491786 28535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:31:47.502357 28535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:31:47.504133 28535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:31:47.504287 28535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:31:47.504511 28535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:31:47.504792 28535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:31:47.504854 28535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.504894 28535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:31:47.504954 28535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:31:47.510641 28535 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40143
I20260929 06:31:47.510689 28663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40143 every 8 connection(s)
I20260929 06:31:47.511008 28535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:31:47.511961 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.511623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:59401)
0929 06:31:47.511755 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.511836 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260929 06:31:47.512353 28664 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:44701 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:44701: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
I20260929 06:31:47.512619 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.512539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:60315)
0929 06:31:47.512580 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.512604 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 06:31:47.512931 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:47.512841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:37959)
0929 06:31:47.512884 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:47.512911 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 06:31:47.512936 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28535
W20260929 06:31:47.513043 28664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:31:48.323104 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.322871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:33751)
0929 06:31:48.322998 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:48.323039 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260929 06:31:48.393821 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.393622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:41193)
0929 06:31:48.393742 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:48.393778 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260929 06:31:48.513576 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.513342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:48191)
0929 06:31:48.513489 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:48.513525 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":56,"threads_started":1}
I20260929 06:31:48.816861 28671 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:48.817062 28671 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:48.817418 28671 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
Sep 29 06:31:48 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.130@KRBTEST.COM
Sep 29 06:31:48 dist-test-slave-sm0p krb5kdc[27101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663507, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.131@KRBTEST.COM
I20260929 06:31:48.834916 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.817751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:39909 (local address 127.13.15.131:40143)
0929 06:31:48.817898 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:48.817908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:48.817935 (+ 27us) server_negotiation.cc:407] Connection header received
0929 06:31:48.818445 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:48.818448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:48.818487 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:48.818648 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:48.819377 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.820177 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821168 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.821585 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821724 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:48.831473 (+ 9749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:48.831477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:48.831477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:48.831497 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:48.833247 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833636 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:48.833640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:48.833640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:48.833737 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833996 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:48.833996 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:48.833997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:48.834314 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:48.834403 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:48.834667 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 06:31:48.834795 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260929 06:31:48.834903 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.817620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40143 (local address 127.13.15.129:39909)
0929 06:31:48.817679 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:48.817690 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:48.817746 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:48.818693 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:48.818694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:48.819060 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:48.819238 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:48.819245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.820277 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.820278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:48.821073 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:48.821075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.821691 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:48.821808 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:48.829382 (+ 7574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:48.829393 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:48.831354 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:48.833319 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:48.833321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:48.833541 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:48.833806 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:48.833807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:48.833923 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:48.834385 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:48.834391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:48.834507 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 06:31:48.834616 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 06:31:48.834812 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":32}
I20260929 06:31:48.834949 28673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.817740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:37789 (local address 127.13.15.130:43671)
0929 06:31:48.817923 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 06:31:48.817931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:31:48.817957 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:31:48.818441 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:31:48.818444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:31:48.818486 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:31:48.818653 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:31:48.819374 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.820171 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821168 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.821587 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821724 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:48.831129 (+ 9405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:31:48.831134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:31:48.831134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:31:48.831150 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:31:48.833178 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833636 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:48.833640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:48.833640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:48.833738 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833993 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:31:48.833994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:31:48.833995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:31:48.834312 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:31:48.834402 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 06:31:48.834664 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 06:31:48.834831 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260929 06:31:48.835022 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:48.817631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:43671 (local address 127.13.15.129:37789)
0929 06:31:48.817773 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:48.817782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:31:48.817824 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:31:48.818686 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:31:48.818689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:31:48.819063 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:31:48.819238 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:48.819245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.820272 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.820274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:48.821072 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:31:48.821075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:31:48.821687 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:31:48.821688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:31:48.821807 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:31:48.829493 (+ 7686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:31:48.829498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:31:48.830987 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:31:48.833264 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833278 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:48.833280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:48.833539 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:48.833814 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:31:48.833815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:31:48.833815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:31:48.833923 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:31:48.834384 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:31:48.834390 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:31:48.834507 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 06:31:48.834617 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 06:31:48.834892 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":175,"thread_start_us":61,"threads_started":1}
W20260929 06:31:48.835973 28284 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
W20260929 06:31:48.836092 28287 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
I20260929 06:31:48.836148 28287 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03942114d8bc446f85813895870f4041; no voters: 0d907ca2ed68447a949206ecfa95f974, 74546824567a43fca8dfda381c856ef4
I20260929 06:31:48.836252 28671 raft_consensus.cc:2726] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:31:49.323763 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:49.323606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:60817)
0929 06:31:49.323690 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:49.323725 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 06:31:49.394472 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:49.394270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:45163)
0929 06:31:49.394405 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:49.394437 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":55,"threads_started":1}
I20260929 06:31:49.514243 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:49.514027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:46369)
0929 06:31:49.514170 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:49.514207 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260929 06:31:50.324421 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:50.324229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:46027)
0929 06:31:50.324343 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:50.324375 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260929 06:31:50.395144 28679 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:50.395187 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:50.394933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:57869)
0929 06:31:50.395114 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:50.395152 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260929 06:31:50.395241 28679 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:50.395473 28679 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
W20260929 06:31:50.395855 28284 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
W20260929 06:31:50.395916 28287 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
I20260929 06:31:50.395998 28287 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03942114d8bc446f85813895870f4041; no voters: 0d907ca2ed68447a949206ecfa95f974, 74546824567a43fca8dfda381c856ef4
I20260929 06:31:50.396078 28679 raft_consensus.cc:2726] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:31:50.514911 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:50.514720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:42069)
0929 06:31:50.514836 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:50.514873 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260929 06:31:51.325103 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:51.324909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:43135)
0929 06:31:51.325025 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:51.325056 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260929 06:31:51.395913 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:51.395676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:59925)
0929 06:31:51.395809 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:51.395869 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260929 06:31:51.515584 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:51.515393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:60287)
0929 06:31:51.515512 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:51.515546 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260929 06:31:52.325809 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:52.325604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:48709)
0929 06:31:52.325733 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:52.325764 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260929 06:31:52.396474 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:52.396295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:54919)
0929 06:31:52.396404 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:52.396436 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260929 06:31:52.442379 28686 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:52.442474 28686 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:52.442694 28686 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
W20260929 06:31:52.443125 28287 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
W20260929 06:31:52.443184 28284 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
I20260929 06:31:52.443280 28284 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03942114d8bc446f85813895870f4041; no voters: 0d907ca2ed68447a949206ecfa95f974, 74546824567a43fca8dfda381c856ef4
I20260929 06:31:52.443364 28686 raft_consensus.cc:2726] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:31:52.516363 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:52.516165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:37383)
0929 06:31:52.516284 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:52.516322 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260929 06:31:53.326562 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:53.326366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:53119)
0929 06:31:53.326482 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:53.326519 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260929 06:31:53.397058 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:53.396880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:41179)
0929 06:31:53.396986 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:53.397019 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260929 06:31:53.517077 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:53.516864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:33779)
0929 06:31:53.517005 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:53.517037 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260929 06:31:54.327174 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:54.326979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:53367)
0929 06:31:54.327092 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:54.327129 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260929 06:31:54.339303 28692 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:31:54.339386 28692 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:31:54.339578 28692 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
W20260929 06:31:54.340013 28284 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
W20260929 06:31:54.340080 28287 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
I20260929 06:31:54.340127 28287 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03942114d8bc446f85813895870f4041; no voters: 0d907ca2ed68447a949206ecfa95f974, 74546824567a43fca8dfda381c856ef4
I20260929 06:31:54.340231 28692 raft_consensus.cc:2726] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:31:54.397617 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:54.397429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:41777)
0929 06:31:54.397545 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:54.397578 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260929 06:31:54.517729 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:54.517508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:43051)
0929 06:31:54.517633 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:54.517671 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260929 06:31:55.327782 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:55.327574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:50533)
0929 06:31:55.327694 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:55.327727 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260929 06:31:55.398278 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:55.398038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:52923)
0929 06:31:55.398207 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:55.398240 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
I20260929 06:31:55.518486 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:55.518284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:55353)
0929 06:31:55.518401 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:55.518442 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260929 06:31:56.328428 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:56.328199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:52079)
0929 06:31:56.328344 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:56.328381 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260929 06:31:56.398962 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:56.398753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:57169)
0929 06:31:56.398886 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:56.398920 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260929 06:31:56.519164 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:56.518944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:56941)
0929 06:31:56.519080 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:56.519117 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260929 06:31:57.329236 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:57.328977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:47897)
0929 06:31:57.329150 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:57.329187 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260929 06:31:57.399670 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:57.399452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:49217)
0929 06:31:57.399589 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:57.399626 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260929 06:31:57.519878 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:57.519654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:37517)
0929 06:31:57.519795 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:57.519836 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260929 06:31:58.329993 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:58.329734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:60371)
0929 06:31:58.329900 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:58.329946 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260929 06:31:58.400471 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:58.400264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:51287)
0929 06:31:58.400396 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:58.400438 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":57,"threads_started":1}
I20260929 06:31:58.520759 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:58.520529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:34445)
0929 06:31:58.520670 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:58.520716 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260929 06:31:59.330588 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:59.330394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:48261)
0929 06:31:59.330504 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:59.330540 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260929 06:31:59.401095 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:59.400905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:54577)
0929 06:31:59.401022 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:59.401058 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260929 06:31:59.521401 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:31:59.521207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:53365)
0929 06:31:59.521317 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 06:31:59.521351 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260929 06:32:00.331216 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:00.330981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:41945)
0929 06:32:00.331126 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:00.331161 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260929 06:32:00.401866 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:00.401662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:58133)
0929 06:32:00.401794 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:00.401826 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260929 06:32:00.522014 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:00.521821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:34131)
0929 06:32:00.521942 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:00.521973 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260929 06:32:01.331849 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:01.331630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:48223)
0929 06:32:01.331771 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:01.331805 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260929 06:32:01.402539 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:01.402347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:60641)
0929 06:32:01.402467 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:01.402500 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":53,"threads_started":1}
I20260929 06:32:01.522718 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:01.522450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:57971)
0929 06:32:01.522605 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:01.522659 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260929 06:32:01.831645 28716 raft_consensus.cc:482] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:32:01.831736 28716 raft_consensus.cc:504] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d907ca2ed68447a949206ecfa95f974" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 43671 } } peers { permanent_uuid: "74546824567a43fca8dfda381c856ef4" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40143 } } peers { permanent_uuid: "03942114d8bc446f85813895870f4041" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41461 } }
I20260929 06:32:01.831880 28716 leader_election.cc:288] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671), 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143)
W20260929 06:32:01.832304 28284 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d907ca2ed68447a949206ecfa95f974 (127.13.15.130:43671): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
W20260929 06:32:01.832396 28287 leader_election.cc:341] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74546824567a43fca8dfda381c856ef4 (127.13.15.131:40143): Not found: Tablet not found: 977077f357224202a960bd6395ceb647
I20260929 06:32:01.832489 28287 leader_election.cc:302] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03942114d8bc446f85813895870f4041; no voters: 0d907ca2ed68447a949206ecfa95f974, 74546824567a43fca8dfda381c856ef4
I20260929 06:32:01.832588 28716 raft_consensus.cc:2726] T 977077f357224202a960bd6395ceb647 P 03942114d8bc446f85813895870f4041 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 06:32:02.332549 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:02.332307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.129:55639)
0929 06:32:02.332443 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:02.332479 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260929 06:32:02.403191 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:02.403013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.130:48985)
0929 06:32:02.403124 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:02.403154 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260929 06:32:02.516309 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28239
I20260929 06:32:02.520954 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28403
I20260929 06:32:02.523382 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:02.523197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44701 (local address 127.13.15.131:43691)
0929 06:32:02.523315 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:02.523346 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:44701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260929 06:32:02.525147 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28535
2026-09-29T06:32:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18267 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:32:02 dist-test-slave-sm0p krb5kdc[28722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:32:02 dist-test-slave-sm0p krb5kdc[28722](info): set up 2 sockets
Sep 29 06:32:02 dist-test-slave-sm0p krb5kdc[28722](info): commencing operation
krb5kdc: starting...
W20260929 06:32:03.554071 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:32:03 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:32:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:32:03.666999 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:03.745699 28735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:03.745939 28735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:03.745977 28735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:03.747187 28735 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 06:32:03.747267 28735 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 06:32:03.747287 28735 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 06:32:03.747305 28735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:32:03.747339 28735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:03.747370 28735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:03.747407 28735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:32:03.747440 28735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:32:03.748878 28735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:03.749142 28735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:03.749358 28735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:03.754386 28741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:03.754398 28743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:03.754396 28740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:03.754596 28735 server_base.cc:1034] running on GCE node
I20260929 06:32:03.754880 28735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:03.755153 28735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:03.756311 28735 hybrid_clock.cc:648] HybridClock initialized: now 1790663523756308 us; error 37 us; skew 500 ppm
Sep 29 06:32:03 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663523, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:03.758622 28735 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:32:03.759070 28735 webserver.cc:460] Webserver started at http://127.13.15.190:42743/ using document root <none> and password file <none>
I20260929 06:32:03.759282 28735 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:03.759348 28735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:03.759516 28735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:32:03.760484 28735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "2a146513f20048f4af8bc341a9c2dee2"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "262cd24d138c3e71c1ee90d2ff10ea82"
server_key_iv: "74ae59a34842044bee52bcd417cca5d8"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.760831 28735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "2a146513f20048f4af8bc341a9c2dee2"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "262cd24d138c3e71c1ee90d2ff10ea82"
server_key_iv: "74ae59a34842044bee52bcd417cca5d8"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.762108 28735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:32:03.762825 28750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:03.762984 28735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:32:03.763079 28735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "2a146513f20048f4af8bc341a9c2dee2"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "262cd24d138c3e71c1ee90d2ff10ea82"
server_key_iv: "74ae59a34842044bee52bcd417cca5d8"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.763156 28735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:03.791991 28735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:03.793798 28735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:03.793970 28735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:03.797724 28735 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34499
I20260929 06:32:03.797794 28802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34499 every 8 connection(s)
I20260929 06:32:03.798153 28735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:32:03.799009 28803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:03.801347 28803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap starting.
I20260929 06:32:03.801466 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28735
I20260929 06:32:03.801581 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:32:03.801810 13374 external_mini_cluster.cc:1428] Setting key 0c06f86739a6145bebc4baf8d53ac0a8
I20260929 06:32:03.802102 28803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:03.802505 28803 log.cc:824] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:03.803442 28803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: No bootstrap required, opened a new log
I20260929 06:32:03.805579 28803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:03.805756 28803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:03.805799 28803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Initialized, Role: FOLLOWER
I20260929 06:32:03.805912 28803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:03.805984 28803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:32:03.806018 28803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:32:03.806054 28803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:03.806598 28803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:03.806727 28803 leader_election.cc:302] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a146513f20048f4af8bc341a9c2dee2; no voters:
I20260929 06:32:03.806875 28803 leader_election.cc:288] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:32:03.806913 28808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:32:03.807139 28808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 LEADER]: Becoming Leader. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Running, Role: LEADER
I20260929 06:32:03.807199 28803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:32:03.807281 28808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:03.808832 28808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a146513f20048f4af8bc341a9c2dee2. Latest consensus state: current_term: 1 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:03.809052 28808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
W20260929 06:32:03.809562 28822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:32:03.810899 28822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 29 06:32:03 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:03.810184 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:03.811110 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:32:03.811916 28825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:32:03.812186 28825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:32:03.815171 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:03.802352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.13.15.190:34499)
0929 06:32:03.802559 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:03.802569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:03.802670 (+ 101us) server_negotiation.cc:407] Connection header received
0929 06:32:03.803336 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:03.803346 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:03.803425 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:03.803574 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:03.804381 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.806560 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.807637 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.808274 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.808395 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:03.811485 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:03.811488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:03.811489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:03.811500 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:03.813565 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:03.813904 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:03.813907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:03.813908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:03.814014 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:03.814244 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:03.814245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:03.814245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:03.814473 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:03.814631 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:03.814871 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 06:32:03.815017 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260929 06:32:03.815691 28825 catalog_manager.cc:1347] Generated new cluster ID: c4800c9c987d4b6b86cc23d4c64af9f2
I20260929 06:32:03.815739 28825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:32:03.825686 28825 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:32:03.826323 28825 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:32:03.833997 28825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Generated new TSK 0
I20260929 06:32:03.834206 28825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:32:03.864882 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:03.944403 28829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:03.944614 28829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:03.944638 28829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:03.945696 28829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:03.945739 28829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:03.945765 28829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:32:03.947115 28829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:03.947288 28829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:03.947466 28829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:03.952330 28835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:03.952358 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:03.952360 28837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:03.952687 28829 server_base.cc:1034] running on GCE node
I20260929 06:32:03.952848 28829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:03.953025 28829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:03.954197 28829 hybrid_clock.cc:648] HybridClock initialized: now 1790663523954184 us; error 38 us; skew 500 ppm
Sep 29 06:32:03 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663523, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:03.956521 28829 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:32:03.956926 28829 webserver.cc:460] Webserver started at http://127.13.15.129:41279/ using document root <none> and password file <none>
I20260929 06:32:03.957137 28829 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:03.957221 28829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:03.957369 28829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:32:03.958459 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "62e8196632914032a769a4de48e8e5ad"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "3f8e7b94880ecea568433d4123e398d7"
server_key_iv: "ea4c25c77057df7139d525e27f5abc96"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.958804 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "62e8196632914032a769a4de48e8e5ad"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "3f8e7b94880ecea568433d4123e398d7"
server_key_iv: "ea4c25c77057df7139d525e27f5abc96"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.960139 28829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:32:03.960791 28844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:03.960980 28829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:32:03.961073 28829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "62e8196632914032a769a4de48e8e5ad"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "3f8e7b94880ecea568433d4123e398d7"
server_key_iv: "ea4c25c77057df7139d525e27f5abc96"
server_key_version: "encryptionkey@0"
I20260929 06:32:03.961153 28829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:03.969274 28829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:03.971017 28829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:03.971165 28829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:03.971364 28829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:03.971652 28829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:03.971705 28829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:03.971743 28829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:03.971771 28829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:03.977331 28829 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38511
I20260929 06:32:03.977411 28957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38511 every 8 connection(s)
I20260929 06:32:03.977703 28829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:32:03.979466 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28829
I20260929 06:32:03.979559 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:32:03.979779 13374 external_mini_cluster.cc:1428] Setting key 15a451bea224e48f4269176b09c9b2fd
Sep 29 06:32:03 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663523, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:03.988304 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:03.978431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:52863 (local address 127.13.15.190:34499)
0929 06:32:03.978498 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:03.978500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:03.978675 (+ 175us) server_negotiation.cc:407] Connection header received
0929 06:32:03.979180 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:03.979182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:03.979194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:03.979245 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:03.980367 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.981125 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.982655 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.983000 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.983075 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:03.985256 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:03.985258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:03.985259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:03.985267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:03.986934 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:03.987306 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:03.987307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:03.987308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:03.987372 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:03.987647 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:03.987648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:03.987649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:03.987804 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:03.987886 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:03.988115 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 06:32:03.988206 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:32:03.988339 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:03.978406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:52863)
0929 06:32:03.978557 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:03.978590 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:03.979063 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:03.979312 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:03.979316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:03.980055 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:03.980257 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:03.980262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.981261 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.981262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:03.982581 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:03.982584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:03.983089 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:03.983090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:03.983194 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:03.983727 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:03.983734 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:03.985150 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:03.987013 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:03.987019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:03.987021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:03.987228 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:03.987478 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:03.987478 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:03.987478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:03.987574 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:03.987877 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:03.987881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:03.987989 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 06:32:03.988068 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 06:32:03.988181 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:32:03.988857 28958 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:03.988965 28958 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:03.989152 28958 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:03.989670 28767 ts_manager.cc:194] Registered new tserver with Master: 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511)
I20260929 06:32:03.990342 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:52863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:32:04.005023 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:04.086530 28963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:04.086727 28963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:04.086751 28963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:04.087925 28963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:04.087968 28963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:04.087994 28963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:32:04.089497 28963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:04.089684 28963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:04.089860 28963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:04.094671 28969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:04.094679 28971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:04.094679 28968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:04.094926 28963 server_base.cc:1034] running on GCE node
I20260929 06:32:04.095106 28963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:04.095304 28963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:04.096433 28963 hybrid_clock.cc:648] HybridClock initialized: now 1790663524096402 us; error 43 us; skew 500 ppm
Sep 29 06:32:04 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663524, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:04.098802 28963 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:32:04.099258 28963 webserver.cc:460] Webserver started at http://127.13.15.130:43985/ using document root <none> and password file <none>
I20260929 06:32:04.099480 28963 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:04.099524 28963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:04.099684 28963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:32:04.100694 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "9ff0570dd6fd46dab5595b80fc967061"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "c2219c3ef7f34e6c376d93185ad31eae"
server_key_iv: "de3a1b1cb5ddb2dddfa06b782b663792"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.101046 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "9ff0570dd6fd46dab5595b80fc967061"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "c2219c3ef7f34e6c376d93185ad31eae"
server_key_iv: "de3a1b1cb5ddb2dddfa06b782b663792"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.102582 28963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:32:04.103292 28978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.103470 28963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:32:04.103541 28963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "9ff0570dd6fd46dab5595b80fc967061"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "c2219c3ef7f34e6c376d93185ad31eae"
server_key_iv: "de3a1b1cb5ddb2dddfa06b782b663792"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.103591 28963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:04.113366 28963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:04.115553 28963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:04.115666 28963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:04.115824 28963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:04.116106 28963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:04.116144 28963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.116163 28963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:04.116206 28963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.121699 28963 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:46153
I20260929 06:32:04.121776 29091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:46153 every 8 connection(s)
I20260929 06:32:04.122049 28963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:32:04 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663524, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:04.129607 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 28963
I20260929 06:32:04.129731 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:32:04.129958 13374 external_mini_cluster.cc:1428] Setting key e80bb614ddd964461d47b93270f93484
I20260929 06:32:04.130504 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.122766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60225 (local address 127.13.15.190:34499)
0929 06:32:04.122824 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.122826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.122985 (+ 159us) server_negotiation.cc:407] Connection header received
0929 06:32:04.123411 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.123412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.123420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.123472 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:04.124258 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.124759 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.125655 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.125837 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.125891 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.127759 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:04.127760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:04.127761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:04.127767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:04.129115 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.129442 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.129443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.129443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.129515 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.129788 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.129789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.129790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.129930 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:04.130005 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.130325 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.130415 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:32:04.130636 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.122730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:60225)
0929 06:32:04.122870 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.122908 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.123333 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.123516 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.123519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.124022 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:04.124189 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.124194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.124834 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.124835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.125587 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.125589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.125903 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.125904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.125994 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.126408 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:04.126414 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:04.127677 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:04.129180 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:04.129184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:04.129186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:04.129373 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:04.129596 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:04.129596 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:04.129596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:04.129712 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:04.130008 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:04.130015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:04.130179 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.130282 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.130429 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260929 06:32:04.131055 29092 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:04.131129 29092 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:04.131304 29092 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:04.131646 28767 ts_manager.cc:194] Registered new tserver with Master: 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153)
I20260929 06:32:04.132131 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:60225
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:32:04.153911 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:04.233039 29097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:04.233225 29097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:04.233247 29097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:04.234354 29097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:04.234400 29097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:04.234426 29097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:32:04.235785 29097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:04.235948 29097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:04.236111 29097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:04.240798 29102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:04.240851 29103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:04.240866 29105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:04.241019 29097 server_base.cc:1034] running on GCE node
I20260929 06:32:04.241245 29097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:04.241434 29097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:04.242566 29097 hybrid_clock.cc:648] HybridClock initialized: now 1790663524242544 us; error 36 us; skew 500 ppm
Sep 29 06:32:04 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663524, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:04.244910 29097 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:32:04.245365 29097 webserver.cc:460] Webserver started at http://127.13.15.131:41475/ using document root <none> and password file <none>
I20260929 06:32:04.245606 29097 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:04.245673 29097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:04.245820 29097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:32:04.246892 29097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "c4f247e17e2d48c4a086ac5d62d691db"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "50de3778c37035774b6ead63c703a57f"
server_key_iv: "5ee4868023117ff7c9264b9d6c50b093"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.247244 29097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "c4f247e17e2d48c4a086ac5d62d691db"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "50de3778c37035774b6ead63c703a57f"
server_key_iv: "5ee4868023117ff7c9264b9d6c50b093"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.248515 29097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 06:32:04.249189 29112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.249357 29097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:32:04.249460 29097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "c4f247e17e2d48c4a086ac5d62d691db"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "50de3778c37035774b6ead63c703a57f"
server_key_iv: "5ee4868023117ff7c9264b9d6c50b093"
server_key_version: "encryptionkey@0"
I20260929 06:32:04.249538 29097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:04.268987 29097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:04.270788 29097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:04.270947 29097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:04.271165 29097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:04.271473 29097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:04.271525 29097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.271556 29097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:04.271584 29097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:04.277482 29097 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40305
I20260929 06:32:04.277531 29225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40305 every 8 connection(s)
I20260929 06:32:04.277808 29097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:32:04.278340 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29097
I20260929 06:32:04.278457 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:32:04.278677 13374 external_mini_cluster.cc:1428] Setting key 7af41d52e95a1f5d61448749ed298f55
Sep 29 06:32:04 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663524, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:04.286736 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.278511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:45141 (local address 127.13.15.190:34499)
0929 06:32:04.278555 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.278557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.278797 (+ 240us) server_negotiation.cc:407] Connection header received
0929 06:32:04.279490 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.279491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.279502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.279546 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:04.280424 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.280922 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.281921 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.282112 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.282249 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.284240 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:04.284242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:04.284242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:04.284251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:04.285358 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.285772 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.285773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.285773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.285831 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.286082 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.286083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.286083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.286241 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:04.286308 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.286569 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.286648 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:32:04.286924 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.278494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:45141)
0929 06:32:04.278669 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.278703 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.279374 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.279609 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.279613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.280147 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:04.280317 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.280321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.281007 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.281008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.281813 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.281815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.282259 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.282260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.282391 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.282808 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:04.282815 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:04.284146 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:04.285461 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:04.285467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:04.285469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:04.285683 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:04.285900 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:04.285901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:04.285902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:04.286002 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:04.286311 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:04.286318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:04.286446 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.286549 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.286697 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260929 06:32:04.287364 29226 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:04.287451 29226 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:04.287608 29226 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:04.287957 28767 ts_manager.cc:194] Registered new tserver with Master: c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305)
I20260929 06:32:04.288326 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:45141
I20260929 06:32:04.290395 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:32:04.297557 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.291800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.13.15.190:34499)
0929 06:32:04.291868 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.291870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.291968 (+ 98us) server_negotiation.cc:407] Connection header received
0929 06:32:04.292044 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.292044 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.292054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.292108 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:04.292710 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.293190 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.293942 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.294167 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.294223 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.295294 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:04.295296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:04.295296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:04.295303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:04.296397 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.296737 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.296738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.296738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.296789 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:04.296994 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:04.296995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:04.296996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:04.297137 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:04.297236 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.297375 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.297461 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 06:32:04.298974 28767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:32:04.300508 28767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:32:04.306675 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.13.15.131:40305)
0929 06:32:04.302980 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.302985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.302996 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:32:04.303066 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.303068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.303086 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.303186 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:04.303392 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304107 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.305536 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306298 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.306390 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.306449 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.306557 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.306593 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260929 06:32:04.306671 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40305 (local address 127.0.0.1:44554)
0929 06:32:04.302961 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.302969 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.303005 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.303253 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.303254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.303257 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:04.303311 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.303313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304216 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.304217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.305453 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.305456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306380 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.306381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.306455 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.306469 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.306542 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.306592 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":11,"thread_start_us":67,"threads_started":1}
I20260929 06:32:04.306910 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.0.0.1:49378)
0929 06:32:04.302750 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.302787 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.302854 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.303422 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.303422 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.303425 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:04.303460 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.303461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304728 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.304729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.305740 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.305743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306672 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.306673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.306738 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.306742 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.306772 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.306816 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260929 06:32:04.307022 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46153 (local address 127.0.0.1:39750)
0929 06:32:04.302822 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.302831 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.302884 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.303187 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.303190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.303192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:04.303256 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.303258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304596 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.304597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.305637 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.305639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306724 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.306725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.306782 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.306785 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.306864 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.306914 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":23,"thread_start_us":61,"threads_started":1}
I20260929 06:32:04.307125 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.13.15.130:46153)
0929 06:32:04.302980 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.302985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.302996 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:32:04.303037 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.303039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.303055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.303178 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:04.303346 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304463 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.305717 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306569 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.306643 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.306697 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.306958 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.307006 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260929 06:32:04.307288 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.302655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.13.15.129:38511)
0929 06:32:04.303001 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.303006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.303016 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:32:04.303277 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.303284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.303305 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.303528 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:04.303599 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.304627 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.305814 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.306580 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.307024 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.307090 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.307115 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.307177 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":86,"threads_started":1}
I20260929 06:32:04.307817 28892 tablet_service.cc:1467] Processing CreateTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 (DEFAULT_TABLE table=test-table [id=d67a77db332e4e208bdf9409b510a371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:04.307818 29026 tablet_service.cc:1467] Processing CreateTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 (DEFAULT_TABLE table=test-table [id=d67a77db332e4e208bdf9409b510a371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:04.308168 28892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92faeeb7ced14cfa993e86b7e25e06a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:04.308168 29026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92faeeb7ced14cfa993e86b7e25e06a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:04.308106 29160 tablet_service.cc:1467] Processing CreateTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 (DEFAULT_TABLE table=test-table [id=d67a77db332e4e208bdf9409b510a371]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:04.308454 29160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92faeeb7ced14cfa993e86b7e25e06a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:04.311339 29240 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Bootstrap starting.
I20260929 06:32:04.311604 29242 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Bootstrap starting.
I20260929 06:32:04.312140 29240 tablet_bootstrap.cc:654] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:04.312202 29242 tablet_bootstrap.cc:654] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:04.312733 29240 log.cc:824] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:04.312747 29242 log.cc:824] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:04.313048 29241 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Bootstrap starting.
I20260929 06:32:04.313562 29240 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: No bootstrap required, opened a new log
I20260929 06:32:04.313558 29242 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: No bootstrap required, opened a new log
I20260929 06:32:04.313616 29240 ts_tablet_manager.cc:1397] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:32:04.313668 29241 tablet_bootstrap.cc:654] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:04.313720 29242 ts_tablet_manager.cc:1397] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:32:04.314380 29241 log.cc:824] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:04.315097 29241 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: No bootstrap required, opened a new log
I20260929 06:32:04.315162 29241 ts_tablet_manager.cc:1397] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:32:04.315510 29240 raft_consensus.cc:348] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.315662 29240 raft_consensus.cc:374] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:04.315723 29240 raft_consensus.cc:729] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f247e17e2d48c4a086ac5d62d691db, State: Initialized, Role: FOLLOWER
I20260929 06:32:04.315834 29240 consensus_queue.cc:260] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.315764 29242 raft_consensus.cc:348] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.315933 29242 raft_consensus.cc:374] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:04.315979 29242 raft_consensus.cc:729] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff0570dd6fd46dab5595b80fc967061, State: Initialized, Role: FOLLOWER
I20260929 06:32:04.316128 29240 ts_tablet_manager.cc:1428] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:32:04.316097 29242 consensus_queue.cc:260] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.316262 29226 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:04.316342 29242 ts_tablet_manager.cc:1428] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:32:04.316468 29092 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:04.317147 29241 raft_consensus.cc:348] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.317312 29241 raft_consensus.cc:374] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:04.317364 29241 raft_consensus.cc:729] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e8196632914032a769a4de48e8e5ad, State: Initialized, Role: FOLLOWER
I20260929 06:32:04.317480 29241 consensus_queue.cc:260] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.317696 29241 ts_tablet_manager.cc:1428] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:32:04.317770 28958 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:04.366329 29247 raft_consensus.cc:482] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:32:04.366472 29247 raft_consensus.cc:504] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.366921 29247 leader_election.cc:288] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511), c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305)
I20260929 06:32:04.369634 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.367127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:46319 (local address 127.13.15.129:38511)
0929 06:32:04.367185 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.367187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.367195 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:32:04.367224 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.367225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.367235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.367308 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:04.367465 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.367954 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.368892 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.369278 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.369368 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.369378 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.369533 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.369569 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:32:04.369634 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.367046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.13.15.130:46319)
0929 06:32:04.367090 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.367102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.367147 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.367310 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.367311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.367312 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:04.367396 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.367398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.368062 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.368063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.368792 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.368794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.369367 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.369368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.369448 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.369451 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.369525 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.369565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260929 06:32:04.369701 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.367177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:60513 (local address 127.13.15.131:40305)
0929 06:32:04.367227 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.367229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.367303 (+ 74us) server_negotiation.cc:407] Connection header received
0929 06:32:04.367369 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.367370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.367379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.367431 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:04.367604 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.368063 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.368970 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.369343 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.369434 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.369442 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.369596 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.369629 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:32:04.369776 29249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.367059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40305 (local address 127.13.15.130:60513)
0929 06:32:04.367239 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.367247 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:04.367317 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:04.367448 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:04.367448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:04.367450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:04.367520 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.367522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.368149 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.368150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.368881 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:04.368884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.369435 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.369436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:04.369513 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.369516 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:32:04.369588 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 06:32:04.369626 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260929 06:32:04.370021 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92faeeb7ced14cfa993e86b7e25e06a4" candidate_uuid: "9ff0570dd6fd46dab5595b80fc967061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e8196632914032a769a4de48e8e5ad" is_pre_election: true
I20260929 06:32:04.370101 29180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92faeeb7ced14cfa993e86b7e25e06a4" candidate_uuid: "9ff0570dd6fd46dab5595b80fc967061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f247e17e2d48c4a086ac5d62d691db" is_pre_election: true
I20260929 06:32:04.370190 28912 raft_consensus.cc:2454] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ff0570dd6fd46dab5595b80fc967061 in term 0.
I20260929 06:32:04.370301 29180 raft_consensus.cc:2454] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ff0570dd6fd46dab5595b80fc967061 in term 0.
I20260929 06:32:04.370434 28982 leader_election.cc:302] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e8196632914032a769a4de48e8e5ad, 9ff0570dd6fd46dab5595b80fc967061; no voters:
I20260929 06:32:04.370612 29247 raft_consensus.cc:2781] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:32:04.370692 29247 raft_consensus.cc:482] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:32:04.370726 29247 raft_consensus.cc:3037] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:04.371431 29247 raft_consensus.cc:504] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.371580 29247 leader_election.cc:288] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [CANDIDATE]: Term 1 election: Requested vote from peers 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511), c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305)
I20260929 06:32:04.371716 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92faeeb7ced14cfa993e86b7e25e06a4" candidate_uuid: "9ff0570dd6fd46dab5595b80fc967061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e8196632914032a769a4de48e8e5ad"
I20260929 06:32:04.371739 29180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92faeeb7ced14cfa993e86b7e25e06a4" candidate_uuid: "9ff0570dd6fd46dab5595b80fc967061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f247e17e2d48c4a086ac5d62d691db"
I20260929 06:32:04.371807 28912 raft_consensus.cc:3037] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:04.371806 29180 raft_consensus.cc:3037] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:04.372516 29180 raft_consensus.cc:2454] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ff0570dd6fd46dab5595b80fc967061 in term 1.
I20260929 06:32:04.372613 28912 raft_consensus.cc:2454] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ff0570dd6fd46dab5595b80fc967061 in term 1.
W20260929 06:32:04.372605 29093 tablet.cc:2369] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:32:04.372692 28981 leader_election.cc:302] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ff0570dd6fd46dab5595b80fc967061, c4f247e17e2d48c4a086ac5d62d691db; no voters:
I20260929 06:32:04.372844 29247 raft_consensus.cc:2781] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:32:04.372997 29247 raft_consensus.cc:686] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 1 LEADER]: Becoming Leader. State: Replica: 9ff0570dd6fd46dab5595b80fc967061, State: Running, Role: LEADER
I20260929 06:32:04.373091 29247 consensus_queue.cc:237] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:04.373847 28766 catalog_manager.cc:5495] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130). New cstate: current_term: 1 leader_uuid: "9ff0570dd6fd46dab5595b80fc967061" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } health_report { overall_health: HEALTHY } } }
I20260929 06:32:04.379015 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.376628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.13.15.130:46153)
0929 06:32:04.376674 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:04.376676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:04.376686 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:32:04.376763 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:04.376763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:04.376780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:04.376864 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:32:04.377010 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.377466 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.378242 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:04.378445 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:04.378521 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:04.378689 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:32:04.378788 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:32:04.378880 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:04.378919 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 06:32:04.378938 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:32:04.380566 29180 raft_consensus.cc:1261] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Refusing update from remote peer 9ff0570dd6fd46dab5595b80fc967061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:32:04.380666 28912 raft_consensus.cc:1261] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Refusing update from remote peer 9ff0570dd6fd46dab5595b80fc967061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:32:04.380908 29247 consensus_queue.cc:1035] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:32:04.381065 29250 consensus_queue.cc:1035] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:32:04.383432 29254 mvcc.cc:205] Tried to move back new op lower bound from 7334557795860475904 to 7334557795832664064. Current Snapshot: MvccSnapshot[applied={T|T < 7334557795860475904}]
I20260929 06:32:04.383893 29255 mvcc.cc:205] Tried to move back new op lower bound from 7334557795860475904 to 7334557795832664064. Current Snapshot: MvccSnapshot[applied={T|T < 7334557795860475904}]
I20260929 06:32:04.385915 28766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:32:04.386003 28766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:32:04.386744 28766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Sending DeleteTablet for 3 replicas of tablet 92faeeb7ced14cfa993e86b7e25e06a4
I20260929 06:32:04.387094 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28829
I20260929 06:32:04.387068 28892 tablet_service.cc:1514] Processing DeleteTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:49378
I20260929 06:32:04.387069 29160 tablet_service.cc:1514] Processing DeleteTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260929 06:32:04.387959 29026 tablet_service.cc:1514] Processing DeleteTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:39750
I20260929 06:32:04.388013 29263 tablet_replica.cc:330] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: stopping tablet replica
I20260929 06:32:04.388110 29263 raft_consensus.cc:2229] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:32:04.388214 29263 raft_consensus.cc:2258] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:04.388417 29264 tablet_replica.cc:330] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: stopping tablet replica
I20260929 06:32:04.388509 29264 raft_consensus.cc:2229] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:32:04.388624 29264 raft_consensus.cc:2258] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:04.389043 29264 ts_tablet_manager.cc:1905] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:32:04.390452 29263 ts_tablet_manager.cc:1905] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:32:04.390981 29264 ts_tablet_manager.cc:1918] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:32:04.391057 29264 log.cc:1196] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/92faeeb7ced14cfa993e86b7e25e06a4
I20260929 06:32:04.391337 29264 ts_tablet_manager.cc:1939] T 92faeeb7ced14cfa993e86b7e25e06a4 P 9ff0570dd6fd46dab5595b80fc967061: Deleting consensus metadata
I20260929 06:32:04.391813 28752 catalog_manager.cc:4841] TS 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153): tablet 92faeeb7ced14cfa993e86b7e25e06a4 (table test-table [id=d67a77db332e4e208bdf9409b510a371]) successfully deleted
I20260929 06:32:04.392184 29263 ts_tablet_manager.cc:1918] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:32:04.392259 29263 log.cc:1196] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/92faeeb7ced14cfa993e86b7e25e06a4
I20260929 06:32:04.392467 29263 ts_tablet_manager.cc:1939] T 92faeeb7ced14cfa993e86b7e25e06a4 P c4f247e17e2d48c4a086ac5d62d691db: Deleting consensus metadata
I20260929 06:32:04.393069 28753 catalog_manager.cc:4841] TS c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305): tablet 92faeeb7ced14cfa993e86b7e25e06a4 (table test-table [id=d67a77db332e4e208bdf9409b510a371]) successfully deleted
W20260929 06:32:04.394320 28754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:38511 (error 108)
I20260929 06:32:04.394811 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:04.394682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.0.0.1:49122)
0929 06:32:04.394743 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:04.394778 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 06:32:04.394876 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28963
W20260929 06:32:04.395604 28754 catalog_manager.cc:4584] TS 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92faeeb7ced14cfa993e86b7e25e06a4: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38511: connect: Connection refused (error 111)
I20260929 06:32:04.399827 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29097
I20260929 06:32:04.404191 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 28735
I20260929 06:32:24.409279 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--webserver_interface=127.13.15.190
--webserver_port=42743
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:24.487725 29266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:24.487926 29266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:24.487949 29266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:24.489078 29266 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 06:32:24.489130 29266 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 06:32:24.489142 29266 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 06:32:24.489153 29266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:32:24.489190 29266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:24.489220 29266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:24.489236 29266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:32:24.489260 29266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:32:24.490778 29266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=42743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:24.490998 29266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:24.491240 29266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:24.499799 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.499874 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.499763 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.500034 29266 server_base.cc:1034] running on GCE node
I20260929 06:32:24.500276 29266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:24.500581 29266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:24.501807 29266 hybrid_clock.cc:648] HybridClock initialized: now 1790663544501767 us; error 43 us; skew 500 ppm
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:24.504371 29266 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:32:24.504842 29266 webserver.cc:460] Webserver started at http://127.13.15.190:42743/ using document root <none> and password file <none>
I20260929 06:32:24.505075 29266 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:24.505180 29266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:24.506675 29266 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:32:24.507308 29282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.507462 29266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:32:24.507561 29266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "2a146513f20048f4af8bc341a9c2dee2"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "262cd24d138c3e71c1ee90d2ff10ea82"
server_key_iv: "74ae59a34842044bee52bcd417cca5d8"
server_key_version: "encryptionkey@0"
I20260929 06:32:24.507954 29266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:24.532081 29266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:24.534235 29266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:24.534379 29266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:24.538381 29266 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34499
I20260929 06:32:24.538422 29334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34499 every 8 connection(s)
I20260929 06:32:24.538807 29266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:32:24.540208 29335 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:32:24.540865 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap starting.
I20260929 06:32:24.544020 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29266
I20260929 06:32:24.544363 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:38511
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=41279
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:32:24.549981 29335 log.cc:824] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:24.552753 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:24.553095 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap complete.
I20260929 06:32:24.556661 29335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:24.556957 29335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Initialized, Role: FOLLOWER
I20260929 06:32:24.557098 29335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:24.557173 29335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:32:24.557229 29335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:32:24.557279 29335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:32:24.558548 29335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:24.558669 29335 leader_election.cc:302] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a146513f20048f4af8bc341a9c2dee2; no voters:
I20260929 06:32:24.558835 29335 leader_election.cc:288] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 06:32:24.558990 29339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 06:32:24.559095 29335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:32:24.559250 29339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 LEADER]: Becoming Leader. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Running, Role: LEADER
I20260929 06:32:24.559450 29339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:24.559795 29339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a146513f20048f4af8bc341a9c2dee2. Latest consensus state: current_term: 2 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:24.559795 29341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:24.560009 29341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:32:24.560407 29339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:32:24.560998 29355 catalog_manager.cc:1259] Loaded cluster ID: c4800c9c987d4b6b86cc23d4c64af9f2
I20260929 06:32:24.561055 29355 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: loading cluster ID for follower catalog manager: success
I20260929 06:32:24.562322 29355 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: acquiring CA information for follower catalog manager: success
I20260929 06:32:24.562779 29355 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:32:24.562875 29351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:32:24.563259 29351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:32:24.563376 29351 catalog_manager.cc:1259] Loaded cluster ID: c4800c9c987d4b6b86cc23d4c64af9f2
I20260929 06:32:24.563419 29351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:32:24.563675 29351 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:32:24.563776 29351 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Loaded TSK: 0
I20260929 06:32:24.579342 29351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Generated new TSK 1
I20260929 06:32:24.579459 29351 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:32:24.628548 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:24.628762 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:24.628800 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:24.629971 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:24.630031 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:24.630074 29337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:32:24.631495 29337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:38511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=41279
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:24.631735 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:24.631961 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:24.637785 29363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.637812 29365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.637882 29362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.638224 29337 server_base.cc:1034] running on GCE node
I20260929 06:32:24.638441 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:24.638695 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:24.639870 29337 hybrid_clock.cc:648] HybridClock initialized: now 1790663544639851 us; error 32 us; skew 500 ppm
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:24.642478 29337 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:32:24.642871 29337 webserver.cc:460] Webserver started at http://127.13.15.129:41279/ using document root <none> and password file <none>
I20260929 06:32:24.643103 29337 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:24.643195 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:24.644589 29337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:32:24.645326 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.645505 29337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:32:24.645596 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "62e8196632914032a769a4de48e8e5ad"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "3f8e7b94880ecea568433d4123e398d7"
server_key_iv: "ea4c25c77057df7139d525e27f5abc96"
server_key_version: "encryptionkey@0"
I20260929 06:32:24.645943 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:24.654603 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:24.656869 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:24.657027 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:24.657259 29337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:24.657721 29379 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 06:32:24.659745 29337 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 06:32:24.659895 29337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 06:32:24.659940 29337 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 06:32:24.660610 29337 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 06:32:24.660665 29337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 06:32:24.660742 29379 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Bootstrap starting.
I20260929 06:32:24.667807 29337 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38511
I20260929 06:32:24.668151 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:32:24.669524 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29337
I20260929 06:32:24.669823 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:46153
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=43985
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:32:24.671736 29379 log.cc:824] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:24.674335 29486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38511 every 8 connection(s)
I20260929 06:32:24.674593 29379 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:24.674944 29379 tablet_bootstrap.cc:492] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Bootstrap complete.
I20260929 06:32:24.676159 29379 ts_tablet_manager.cc:1397] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260929 06:32:24.678069 29379 raft_consensus.cc:348] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:24.678596 29379 raft_consensus.cc:729] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e8196632914032a769a4de48e8e5ad, State: Initialized, Role: FOLLOWER
I20260929 06:32:24.678845 29379 consensus_queue.cc:260] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } } peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } }
I20260929 06:32:24.679206 29379 ts_tablet_manager.cc:1428] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:24.690461 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.668950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:53909)
0929 06:32:24.672063 (+ 3113us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.672112 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.678502 (+ 6390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.679172 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.679177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.679609 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:24.679881 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.679888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.681669 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.681670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.682970 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.682973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.683739 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.683740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.683882 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.684472 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:24.684481 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:24.686261 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:24.688548 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.688555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.688557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.688866 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.689172 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.689173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.689173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.689307 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.689744 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:24.689751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:24.689901 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.690007 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.690215 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3069,"thread_start_us":101,"threads_started":1}
I20260929 06:32:24.690573 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.669015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53909 (local address 127.13.15.190:34499)
0929 06:32:24.678331 (+ 9316us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.678342 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.678395 (+ 53us) server_negotiation.cc:407] Connection header received
0929 06:32:24.678796 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.678803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.678865 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.679000 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:24.680034 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.681547 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.683064 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.683651 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.683771 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.686375 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:24.686379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:24.686380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:24.686391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:24.688449 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.688961 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.688964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.688964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.689094 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.689389 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.689390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.689391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.689668 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:24.689746 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.690100 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.690306 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9226,"thread_start_us":69,"threads_started":1}
I20260929 06:32:24.691708 29487 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:24.691807 29487 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:24.691969 29487 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:24.692678 29299 ts_manager.cc:194] Registered new tserver with Master: 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511)
I20260929 06:32:24.693807 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:53909
I20260929 06:32:24.695134 29487 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:24.698884 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.694296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.0.0.1:44762)
0929 06:32:24.694474 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.694511 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.694751 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.695104 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.695107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.695109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:24.695169 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.695171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.696471 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.696472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.697702 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.697705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.698598 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.698599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.698677 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.698690 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.698761 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.698810 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260929 06:32:24.698891 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.694560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.13.15.129:38511)
0929 06:32:24.694874 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.694879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.694888 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:32:24.694918 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.694920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.694980 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.695242 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:24.695289 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.696346 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.697795 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.698500 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.698578 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.698626 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.698786 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.698825 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":93,"threads_started":1}
I20260929 06:32:24.699544 29408 tablet_service.cc:1514] Processing DeleteTablet for tablet 92faeeb7ced14cfa993e86b7e25e06a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:04 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 06:32:24.700093 29496 tablet_replica.cc:330] stopping tablet replica
I20260929 06:32:24.700192 29496 raft_consensus.cc:2229] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:32:24.700264 29496 raft_consensus.cc:2258] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:24.700556 29496 ts_tablet_manager.cc:1905] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:32:24.702790 29496 ts_tablet_manager.cc:1918] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:32:24.702862 29496 log.cc:1196] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal/wals/92faeeb7ced14cfa993e86b7e25e06a4
I20260929 06:32:24.703130 29496 ts_tablet_manager.cc:1939] T 92faeeb7ced14cfa993e86b7e25e06a4 P 62e8196632914032a769a4de48e8e5ad: Deleting consensus metadata
I20260929 06:32:24.703636 29286 catalog_manager.cc:4841] TS 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511): tablet 92faeeb7ced14cfa993e86b7e25e06a4 (table test-table [id=d67a77db332e4e208bdf9409b510a371]) successfully deleted
W20260929 06:32:24.756678 29491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:24.756887 29491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:24.756919 29491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:24.758030 29491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:24.758090 29491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:24.758132 29491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:32:24.759763 29491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:46153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=43985
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:24.759999 29491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:24.760226 29491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:24.765038 29502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.765040 29504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.765158 29491 server_base.cc:1034] running on GCE node
W20260929 06:32:24.765074 29501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.765512 29491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:24.765740 29491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:24.766893 29491 hybrid_clock.cc:648] HybridClock initialized: now 1790663544766889 us; error 35 us; skew 500 ppm
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:24.769631 29491 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:32:24.770121 29491 webserver.cc:460] Webserver started at http://127.13.15.130:43985/ using document root <none> and password file <none>
I20260929 06:32:24.770406 29491 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:24.770470 29491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:24.771728 29491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:32:24.772271 29511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.772424 29491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:32:24.772500 29491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "9ff0570dd6fd46dab5595b80fc967061"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "c2219c3ef7f34e6c376d93185ad31eae"
server_key_iv: "de3a1b1cb5ddb2dddfa06b782b663792"
server_key_version: "encryptionkey@0"
I20260929 06:32:24.772781 29491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:24.779801 29491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:24.781551 29491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:24.781653 29491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:24.781811 29491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:24.782037 29491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:24.782065 29491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.782084 29491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:24.782096 29491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.788002 29491 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:46153
I20260929 06:32:24.788054 29624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:46153 every 8 connection(s)
I20260929 06:32:24.788357 29491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:24.795486 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29491
I20260929 06:32:24.795818 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:40305
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=41475
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:32:24.797488 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.788995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:48767 (local address 127.13.15.190:34499)
0929 06:32:24.789069 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.789071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.789222 (+ 151us) server_negotiation.cc:407] Connection header received
0929 06:32:24.789665 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.789666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.789680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.789731 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:24.790686 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.791224 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.792256 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.792472 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.792532 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.794522 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:24.794523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:24.794524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:24.794530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:24.795930 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.796391 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.796392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.796392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.796462 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.796741 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.796742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.796742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.796860 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:24.796915 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.797247 (+ 332us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.797366 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:32:24.797570 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.788968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:48767)
0929 06:32:24.789116 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.789148 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.789581 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.789777 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.789780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.790375 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:24.790564 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.790568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.791341 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.791342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.792163 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.792165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.792547 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.792547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.792641 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.793039 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:24.793046 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:24.794425 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:24.796026 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.796032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.796034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.796325 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.796525 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.796526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.796526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.796670 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.796930 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:24.796936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:24.797095 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.797215 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.797393 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":93,"threads_started":1}
I20260929 06:32:24.798275 29625 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:24.798377 29625 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:24.798539 29625 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:24.798928 29299 ts_manager.cc:194] Registered new tserver with Master: 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153)
I20260929 06:32:24.799302 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:48767
W20260929 06:32:24.877275 29628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:24.877480 29628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:24.877504 29628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:24.878686 29628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:24.878731 29628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:24.878782 29628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:32:24.880227 29628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:40305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=41475
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:24.880460 29628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:24.880702 29628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:24.885676 29634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:24.885672 29636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.885777 29628 server_base.cc:1034] running on GCE node
W20260929 06:32:24.885923 29633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:24.886214 29628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:24.886418 29628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:24.887575 29628 hybrid_clock.cc:648] HybridClock initialized: now 1790663544887558 us; error 32 us; skew 500 ppm
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:24.890105 29628 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:32:24.890633 29628 webserver.cc:460] Webserver started at http://127.13.15.131:41475/ using document root <none> and password file <none>
I20260929 06:32:24.890887 29628 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:24.890972 29628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:24.892385 29628 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:32:24.892936 29643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.893097 29628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.893222 29628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "c4f247e17e2d48c4a086ac5d62d691db"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "50de3778c37035774b6ead63c703a57f"
server_key_iv: "5ee4868023117ff7c9264b9d6c50b093"
server_key_version: "encryptionkey@0"
I20260929 06:32:24.893606 29628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:24.917148 29628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:24.919672 29628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:24.919852 29628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:24.920101 29628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:24.920428 29628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:24.920481 29628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.920526 29628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:24.920573 29628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.926946 29628 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40305
I20260929 06:32:24.926988 29756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40305 every 8 connection(s)
I20260929 06:32:24.927353 29628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:32:24.931360 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29628
Sep 29 06:32:24 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663544, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:32:24.937266 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.928083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:37391 (local address 127.13.15.190:34499)
0929 06:32:24.928174 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.928177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.928367 (+ 190us) server_negotiation.cc:407] Connection header received
0929 06:32:24.928867 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.928869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.928880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.928940 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:24.929845 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.930364 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.931399 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.931662 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.931749 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.934242 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:24.934243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:24.934243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:24.934252 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:24.935839 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.936323 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.936324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.936325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.936381 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.936606 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.936607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.936607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.936714 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:24.936801 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.937059 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.937162 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 06:32:24.937448 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.928059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:37391)
0929 06:32:24.928239 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.928285 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.928738 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.928994 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.928997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.929557 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:24.929745 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.929751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.930440 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.930441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.931314 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.931316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.931798 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.931799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.931973 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.932510 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:24.932522 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:24.934116 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:24.935932 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.935939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.935941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.936244 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.936442 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:24.936443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:24.936443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:24.936543 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:24.936794 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:24.936800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:24.936929 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.937024 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.937177 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260929 06:32:24.937901 29757 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34499
I20260929 06:32:24.938002 29757 heartbeater.cc:464] Registering TS with master...
I20260929 06:32:24.938215 29757 heartbeater.cc:511] Master 127.13.15.190:34499 requested a full tablet report, sending...
I20260929 06:32:24.938688 29299 ts_manager.cc:194] Registered new tserver with Master: c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305)
I20260929 06:32:24.938920 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.931826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.13.15.190:34499)
0929 06:32:24.931919 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.931920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.931985 (+ 65us) server_negotiation.cc:407] Connection header received
0929 06:32:24.932056 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.932057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.932065 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.932116 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:24.932852 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.933451 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.934355 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.934591 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.934660 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.935952 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:24.935954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:24.935954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:24.935960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:24.937580 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.938042 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.938043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.938043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.938118 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:24.938370 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:24.938371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:24.938371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:24.938521 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:24.938705 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.938749 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.938828 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":53,"threads_started":1}
I20260929 06:32:24.939126 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:37391
I20260929 06:32:24.939656 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:32:24.942080 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.940012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.13.15.190:34499)
0929 06:32:24.940062 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.940064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.940195 (+ 131us) server_negotiation.cc:407] Connection header received
0929 06:32:24.940259 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.940260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.940285 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.940339 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:32:24.940421 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.940840 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.941545 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.941711 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.941759 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.941876 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:32:24.941951 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:32:24.941998 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.942025 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.942039 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 06:32:24.942337 29299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:32:24.943881 29299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:32:24.945812 29408 tablet_service.cc:1467] Processing CreateTablet for tablet 88a2f6944411431f8c683266b4c5469e (DEFAULT_TABLE table=test-table [id=bbdedb375d4b4a238030baf195b91e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:24.946043 29408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a2f6944411431f8c683266b4c5469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:24.947626 29379 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Bootstrap starting.
I20260929 06:32:24.947952 29379 tablet_bootstrap.cc:654] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:24.948479 29379 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: No bootstrap required, opened a new log
I20260929 06:32:24.948545 29379 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 06:32:24.948673 29379 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.948748 29379 raft_consensus.cc:374] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:24.948807 29379 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e8196632914032a769a4de48e8e5ad, State: Initialized, Role: FOLLOWER
I20260929 06:32:24.948850 29379 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.948942 29379 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:24.949451 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.945511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40305 (local address 127.0.0.1:56504)
0929 06:32:24.945542 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.945552 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.945590 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.946159 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.946160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.946162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:24.946206 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.946207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.947278 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.947279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.948179 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.948181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.949187 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.949188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.949252 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.949255 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.949337 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.949382 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 06:32:24.949478 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.945817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.13.15.131:40305)
0929 06:32:24.945972 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.945977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.945991 (+ 14us) server_negotiation.cc:407] Connection header received
0929 06:32:24.946036 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.946038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.946048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.946163 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:24.946285 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.947195 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.948279 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.949090 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.949180 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.949234 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.949350 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.949401 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260929 06:32:24.950047 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.945569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46153 (local address 127.0.0.1:53998)
0929 06:32:24.945712 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:24.945720 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:24.945755 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:24.946372 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:24.946373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:24.946376 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:24.946427 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.946428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.947706 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.947707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.948908 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:24.948911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.949821 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.949823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:24.949886 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.949889 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:32:24.949945 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 06:32:24.949982 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260929 06:32:24.950064 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:24.945801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.13.15.130:46153)
0929 06:32:24.946084 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:24.946184 (+ 100us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:24.946196 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:32:24.946245 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:24.946247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:24.946269 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:24.946496 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:24.946576 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.947621 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.948996 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:24.949741 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:24.949820 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:24.949871 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:24.949957 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 06:32:24.949990 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":85,"threads_started":1}
I20260929 06:32:24.950078 29691 tablet_service.cc:1467] Processing CreateTablet for tablet 88a2f6944411431f8c683266b4c5469e (DEFAULT_TABLE table=test-table [id=bbdedb375d4b4a238030baf195b91e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:24.950454 29691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a2f6944411431f8c683266b4c5469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:24.950644 29559 tablet_service.cc:1467] Processing CreateTablet for tablet 88a2f6944411431f8c683266b4c5469e (DEFAULT_TABLE table=test-table [id=bbdedb375d4b4a238030baf195b91e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:32:24.950942 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a2f6944411431f8c683266b4c5469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:24.953073 29767 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Bootstrap starting.
I20260929 06:32:24.953399 29768 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Bootstrap starting.
I20260929 06:32:24.953842 29767 tablet_bootstrap.cc:654] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:24.954066 29768 tablet_bootstrap.cc:654] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Neither blocks nor log segments found. Creating new log.
I20260929 06:32:24.954416 29767 log.cc:824] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:24.954599 29768 log.cc:824] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:24.955080 29767 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: No bootstrap required, opened a new log
I20260929 06:32:24.955129 29767 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:32:24.955296 29768 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: No bootstrap required, opened a new log
I20260929 06:32:24.955367 29768 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 06:32:24.956453 29767 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.956583 29767 raft_consensus.cc:374] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:24.956604 29767 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f247e17e2d48c4a086ac5d62d691db, State: Initialized, Role: FOLLOWER
I20260929 06:32:24.956719 29767 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.956637 29768 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.956795 29768 raft_consensus.cc:374] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:32:24.956831 29768 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff0570dd6fd46dab5595b80fc967061, State: Initialized, Role: FOLLOWER
I20260929 06:32:24.956943 29768 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:24.956980 29767 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:32:24.957142 29757 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:24.957239 29768 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:32:24.957383 29625 heartbeater.cc:503] Master 127.13.15.190:34499 was elected leader, sending a full tablet report...
I20260929 06:32:25.005304 29771 raft_consensus.cc:482] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:32:25.005476 29771 raft_consensus.cc:504] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:25.005904 29771 leader_election.cc:288] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153), 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511)
I20260929 06:32:25.010244 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.006376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:40977 (local address 127.13.15.130:46153)
0929 06:32:25.006446 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:25.006449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:25.006684 (+ 235us) server_negotiation.cc:407] Connection header received
0929 06:32:25.006756 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:25.006758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:25.006782 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:25.006853 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:25.006986 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.007656 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.009240 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.009867 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.009939 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.009948 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:25.010102 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 06:32:25.010172 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 06:32:25.010270 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.006112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46153 (local address 127.13.15.131:40977)
0929 06:32:25.006566 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:25.006578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:25.006658 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:25.006862 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:25.006863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:25.006865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:25.006919 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:25.006920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.007735 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.007736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:25.009156 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:25.009158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.009947 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.009948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:25.010031 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.010035 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:32:25.010081 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:32:25.010193 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":314,"thread_start_us":137,"threads_started":1}
I20260929 06:32:25.010598 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.006099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.13.15.131:33945)
0929 06:32:25.006226 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:25.006239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:25.006298 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:25.006523 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:25.006524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:25.006526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:32:25.006655 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:25.006657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.007707 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.007708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:25.009153 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:25.009156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.010371 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.010372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:25.010445 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.010449 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:32:25.010501 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 06:32:25.010536 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":141}
I20260929 06:32:25.010684 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.006181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:33945 (local address 127.13.15.129:38511)
0929 06:32:25.006245 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:25.006247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:25.006316 (+ 69us) server_negotiation.cc:407] Connection header received
0929 06:32:25.006359 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:25.006360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:25.006371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:25.006441 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:32:25.006769 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.007579 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.009243 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.010293 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.010372 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.010381 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:25.010567 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 06:32:25.010619 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:32:25.010664 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "c4f247e17e2d48c4a086ac5d62d691db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff0570dd6fd46dab5595b80fc967061" is_pre_election: true
I20260929 06:32:25.010821 29579 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4f247e17e2d48c4a086ac5d62d691db in term 0.
I20260929 06:32:25.010943 29441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "c4f247e17e2d48c4a086ac5d62d691db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e8196632914032a769a4de48e8e5ad" is_pre_election: true
I20260929 06:32:25.011093 29441 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4f247e17e2d48c4a086ac5d62d691db in term 0.
I20260929 06:32:25.011091 29645 leader_election.cc:302] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ff0570dd6fd46dab5595b80fc967061, c4f247e17e2d48c4a086ac5d62d691db; no voters:
I20260929 06:32:25.011224 29771 raft_consensus.cc:2781] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:32:25.011288 29771 raft_consensus.cc:482] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:32:25.011313 29771 raft_consensus.cc:3037] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:25.012012 29771 raft_consensus.cc:504] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:25.012154 29771 leader_election.cc:288] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [CANDIDATE]: Term 1 election: Requested vote from peers 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153), 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511)
I20260929 06:32:25.012274 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "c4f247e17e2d48c4a086ac5d62d691db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff0570dd6fd46dab5595b80fc967061"
I20260929 06:32:25.012284 29441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "c4f247e17e2d48c4a086ac5d62d691db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e8196632914032a769a4de48e8e5ad"
I20260929 06:32:25.012338 29579 raft_consensus.cc:3037] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:25.012390 29441 raft_consensus.cc:3037] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:32:25.013029 29579 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4f247e17e2d48c4a086ac5d62d691db in term 1.
I20260929 06:32:25.013145 29441 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4f247e17e2d48c4a086ac5d62d691db in term 1.
I20260929 06:32:25.013178 29645 leader_election.cc:302] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ff0570dd6fd46dab5595b80fc967061, c4f247e17e2d48c4a086ac5d62d691db; no voters:
I20260929 06:32:25.013263 29771 raft_consensus.cc:2781] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:32:25.013413 29771 raft_consensus.cc:686] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 1 LEADER]: Becoming Leader. State: Replica: c4f247e17e2d48c4a086ac5d62d691db, State: Running, Role: LEADER
I20260929 06:32:25.013510 29771 consensus_queue.cc:237] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:25.014366 29298 catalog_manager.cc:5495] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db reported cstate change: term changed from 0 to 1, leader changed from <none> to c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131). New cstate: current_term: 1 leader_uuid: "c4f247e17e2d48c4a086ac5d62d691db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } health_report { overall_health: UNKNOWN } } }
I20260929 06:32:25.021849 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.020016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.13.15.131:40305)
0929 06:32:25.020061 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:25.020063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:25.020075 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:32:25.020140 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:25.020140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:25.020153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:25.020201 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:32:25.020290 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.020714 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.021339 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.021499 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.021542 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.021660 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:32:25.021698 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 06:32:25.021797 (+ 99us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56514: token expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 06:32:25.021911 29765 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56514: token expired
I20260929 06:32:25.029734 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.022434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.13.15.190:34499)
0929 06:32:25.022485 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:25.022487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:25.022494 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:32:25.022524 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:25.022524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:25.022542 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:25.022620 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:25.023063 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.023799 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.025025 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.025198 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.025241 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.026856 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:25.026858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:25.026858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:25.026865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:25.028526 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:25.028820 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:25.028821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:25.028822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:25.028892 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:25.029110 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:25.029110 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:25.029111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:25.029297 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:25.029414 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:25.029507 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 06:32:25.029604 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:32:25.032678 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.030713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.13.15.131:40305)
0929 06:32:25.030747 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:25.030748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:25.030756 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:32:25.030817 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:25.030817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:25.030825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:25.030860 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:32:25.030948 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.031414 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.032068 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:25.032232 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:25.032291 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:25.032442 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:32:25.032512 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:32:25.032546 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:25.032621 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 06:32:25.032635 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:32:25.034371 29441 raft_consensus.cc:1261] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Refusing update from remote peer c4f247e17e2d48c4a086ac5d62d691db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:32:25.034371 29579 raft_consensus.cc:1261] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Refusing update from remote peer c4f247e17e2d48c4a086ac5d62d691db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:32:25.034703 29774 consensus_queue.cc:1035] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:32:25.034830 29771 consensus_queue.cc:1035] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:32:25.036712 29777 mvcc.cc:205] Tried to move back new op lower bound from 7334557880458252288 to 7334557880376070144. Current Snapshot: MvccSnapshot[applied={T|T < 7334557880458252288}]
I20260929 06:32:25.037245 29779 mvcc.cc:205] Tried to move back new op lower bound from 7334557880458252288 to 7334557880376070144. Current Snapshot: MvccSnapshot[applied={T|T < 7334557880458252288}]
I20260929 06:32:25.037583 29778 mvcc.cc:205] Tried to move back new op lower bound from 7334557880458252288 to 7334557880376070144. Current Snapshot: MvccSnapshot[applied={T|T < 7334557880458252288 or (T in {7334557880458252288})}]
I20260929 06:32:25.039636 29299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:32:25.039722 29299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:32:25.040376 29299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Sending DeleteTablet for 3 replicas of tablet 88a2f6944411431f8c683266b4c5469e
I20260929 06:32:25.040652 29691 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a2f6944411431f8c683266b4c5469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:25 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:56504
I20260929 06:32:25.040668 29408 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a2f6944411431f8c683266b4c5469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:25 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 06:32:25.040750 29559 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a2f6944411431f8c683266b4c5469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:32:25 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:53998
I20260929 06:32:25.040910 29496 tablet_replica.cc:330] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: stopping tablet replica
I20260929 06:32:25.040925 29788 tablet_replica.cc:330] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: stopping tablet replica
I20260929 06:32:25.040973 29496 raft_consensus.cc:2229] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:32:25.041006 29788 raft_consensus.cc:2229] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:32:25.041026 29496 raft_consensus.cc:2258] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:25.041101 29788 raft_consensus.cc:2258] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:25.041247 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29337
I20260929 06:32:25.041350 29496 ts_tablet_manager.cc:1905] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:32:25.042584 29787 tablet_replica.cc:330] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: stopping tablet replica
I20260929 06:32:25.042685 29787 raft_consensus.cc:2229] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:32:25.042804 29787 raft_consensus.cc:2258] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:32:25.043217 29787 ts_tablet_manager.cc:1905] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:32:25.045375 29787 ts_tablet_manager.cc:1918] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:32:25.045485 29787 log.cc:1196] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/88a2f6944411431f8c683266b4c5469e
I20260929 06:32:25.045755 29787 ts_tablet_manager.cc:1939] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Deleting consensus metadata
I20260929 06:32:25.046351 29285 catalog_manager.cc:4841] TS c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305): tablet 88a2f6944411431f8c683266b4c5469e (table test-table [id=bbdedb375d4b4a238030baf195b91e8b]) successfully deleted
W20260929 06:32:25.047971 29286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:38511 (error 108)
I20260929 06:32:25.048269 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29491
I20260929 06:32:25.049373 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.049254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.0.0.1:44764)
0929 06:32:25.049305 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:25.049338 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 06:32:25.049510 29286 catalog_manager.cc:4584] TS 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88a2f6944411431f8c683266b4c5469e: Network error: Client connection negotiation failed: client connection to 127.13.15.129:38511: connect: Connection refused (error 111)
I20260929 06:32:25.052412 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:25.052299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46153 (local address 127.0.0.1:54014)
0929 06:32:25.052348 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:25.052376 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.130:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 06:32:25.052939 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29628
I20260929 06:32:25.057760 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29266
I20260929 06:32:45.063226 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--webserver_interface=127.13.15.190
--webserver_port=42743
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:45.148753 29790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:45.148989 29790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:45.149024 29790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:45.150189 29790 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 06:32:45.150256 29790 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 06:32:45.150279 29790 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 06:32:45.150296 29790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:32:45.150321 29790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:45.150368 29790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:45.150395 29790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:32:45.150424 29790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:32:45.151883 29790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=42743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:45.152148 29790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:45.152370 29790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:45.157444 29798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.157572 29790 server_base.cc:1034] running on GCE node
W20260929 06:32:45.157464 29795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.157452 29796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.158025 29790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:45.158377 29790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:45.159533 29790 hybrid_clock.cc:648] HybridClock initialized: now 1790663565159513 us; error 50 us; skew 500 ppm
Sep 29 06:32:45 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:45.161975 29790 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:32:45.162496 29790 webserver.cc:460] Webserver started at http://127.13.15.190:42743/ using document root <none> and password file <none>
I20260929 06:32:45.162726 29790 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:45.162794 29790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:45.164109 29790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:32:45.164774 29805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.164953 29790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:32:45.165050 29790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "2a146513f20048f4af8bc341a9c2dee2"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "262cd24d138c3e71c1ee90d2ff10ea82"
server_key_iv: "74ae59a34842044bee52bcd417cca5d8"
server_key_version: "encryptionkey@0"
I20260929 06:32:45.165398 29790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:45.182080 29790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:45.183938 29790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:45.184103 29790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:45.188197 29790 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34499
I20260929 06:32:45.188257 29857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34499 every 8 connection(s)
I20260929 06:32:45.188639 29790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:32:45.190083 29858 sys_catalog.cc:263] Verifying existing consensus state
I20260929 06:32:45.190958 29858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap starting.
I20260929 06:32:45.198375 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29790
I20260929 06:32:45.198666 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:38511
--local_ip_for_outbound_sockets=127.13.15.129
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=41279
--webserver_interface=127.13.15.129
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:32:45.199313 29858 log.cc:824] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:45.202795 29858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:45.203159 29858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Bootstrap complete.
I20260929 06:32:45.207175 29858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:45.207578 29858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Initialized, Role: FOLLOWER
I20260929 06:32:45.207772 29858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:45.207861 29858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:32:45.207924 29858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:32:45.207988 29858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 2 FOLLOWER]: Advancing to term 3
I20260929 06:32:45.209604 29858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:45.209735 29858 leader_election.cc:302] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a146513f20048f4af8bc341a9c2dee2; no voters:
I20260929 06:32:45.209918 29858 leader_election.cc:288] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 06:32:45.210013 29862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 06:32:45.210242 29862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [term 3 LEADER]: Becoming Leader. State: Replica: 2a146513f20048f4af8bc341a9c2dee2, State: Running, Role: LEADER
I20260929 06:32:45.210240 29858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:32:45.210393 29862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } }
I20260929 06:32:45.210801 29863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:45.210897 29863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:32:45.211050 29864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a146513f20048f4af8bc341a9c2dee2. Latest consensus state: current_term: 3 leader_uuid: "2a146513f20048f4af8bc341a9c2dee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a146513f20048f4af8bc341a9c2dee2" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34499 } } }
I20260929 06:32:45.211108 29864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:32:45.211215 29868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:32:45.212025 29868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:32:45.212152 29868 catalog_manager.cc:1259] Loaded cluster ID: c4800c9c987d4b6b86cc23d4c64af9f2
I20260929 06:32:45.212419 29868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:32:45.213666 29868 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:32:45.213850 29868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Loaded TSK: 0
I20260929 06:32:45.213898 29868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a146513f20048f4af8bc341a9c2dee2: Loaded TSK: 1
F20260929 06:32:45.214177 29868 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f043ed0d9ad google::LogMessage::Fail() at ??:0
@ 0x7f043ed1088a google::LogMessage::SendToLog() at ??:0
@ 0x7f043ed0d4de google::LogMessage::Flush() at ??:0
@ 0x7f043ed0ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f044147a104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0441477d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0446c671cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0446c6748e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0446c67594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0446c2dc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0446c37ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0440bdb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0440bd0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0441a346db start_thread at ??:0
@ 0x7f043d80b61f clone at ??:0
W20260929 06:32:45.319818 29861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:45.320091 29861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:45.320125 29861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:45.321749 29861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:45.321820 29861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:45.321861 29861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:32:45.324043 29861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:38511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=41279
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:45.324294 29861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:45.324573 29861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:45.329996 29900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.329936 29902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.330231 29861 server_base.cc:1034] running on GCE node
W20260929 06:32:45.329942 29899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.330520 29861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:45.330811 29861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:45.332003 29861 hybrid_clock.cc:648] HybridClock initialized: now 1790663565331989 us; error 29 us; skew 500 ppm
Sep 29 06:32:45 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:45.335114 29861 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:32:45.335681 29861 webserver.cc:460] Webserver started at http://127.13.15.129:41279/ using document root <none> and password file <none>
I20260929 06:32:45.335968 29861 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:45.336045 29861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:45.337867 29861 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 06:32:45.338831 29909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.339015 29861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:32:45.339092 29861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "62e8196632914032a769a4de48e8e5ad"
format_stamp: "Formatted at 2026-09-29 06:32:03 on dist-test-slave-sm0p"
server_key: "3f8e7b94880ecea568433d4123e398d7"
server_key_iv: "ea4c25c77057df7139d525e27f5abc96"
server_key_version: "encryptionkey@0"
I20260929 06:32:45.339435 29861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:45.349901 29861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:45.352121 29861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:45.352259 29861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:45.352500 29861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:45.352960 29916 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 06:32:45.354853 29861 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 06:32:45.354909 29861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 06:32:45.354954 29861 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 06:32:45.355572 29861 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 06:32:45.355625 29861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 06:32:45.355708 29916 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Bootstrap starting.
I20260929 06:32:45.357630 29916 log.cc:824] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:45.360795 29916 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:45.361145 29916 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Bootstrap complete.
I20260929 06:32:45.361306 29916 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 06:32:45.362098 29861 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:38511
I20260929 06:32:45.362231 30024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:38511 every 8 connection(s)
I20260929 06:32:45.362493 29861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:32:45.363343 29916 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:45.363751 29916 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e8196632914032a769a4de48e8e5ad, State: Initialized, Role: FOLLOWER
I20260929 06:32:45.363911 29916 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:45.364200 29916 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:32:45.364442 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 29861
I20260929 06:32:45.364727 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:46153
--local_ip_for_outbound_sockets=127.13.15.130
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=43985
--webserver_interface=127.13.15.130
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:45.436523 30027 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:32:45.363230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:49875)
0929 06:32:45.364827 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.364871 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:45.365034 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:45.436306 (+ 71272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1557,"thread_start_us":83,"threads_started":1}
W20260929 06:32:45.436777 30025 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:34499 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:34499: Client connection negotiation failed: client connection to 127.13.15.190:34499: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 06:32:45.437134 30027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.437025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:46939)
0929 06:32:45.437078 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.437115 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 06:32:45.437497 30027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.437407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:41183)
0929 06:32:45.437450 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.437483 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:32:45.437638 30025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 06:32:45.451359 30029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:45.451545 30029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:45.451568 30029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:45.452795 30029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:45.452845 30029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:45.452874 30029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:32:45.454437 30029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:46153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=43985
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:45.454674 30029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:45.454947 30029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:45.459985 30034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.459946 30037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.459960 30035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.460134 30029 server_base.cc:1034] running on GCE node
I20260929 06:32:45.460399 30029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:45.460651 30029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:45.461807 30029 hybrid_clock.cc:648] HybridClock initialized: now 1790663565461796 us; error 32 us; skew 500 ppm
Sep 29 06:32:45 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:45.464331 30029 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:32:45.464736 30029 webserver.cc:460] Webserver started at http://127.13.15.130:43985/ using document root <none> and password file <none>
I20260929 06:32:45.464968 30029 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:45.465036 30029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:45.466352 30029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:32:45.466974 30044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.467132 30029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.467216 30029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "9ff0570dd6fd46dab5595b80fc967061"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "c2219c3ef7f34e6c376d93185ad31eae"
server_key_iv: "de3a1b1cb5ddb2dddfa06b782b663792"
server_key_version: "encryptionkey@0"
I20260929 06:32:45.467546 30029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:45.480225 30029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:45.482439 30029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:45.482600 30029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:45.482815 30029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:45.483285 30051 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 06:32:45.484637 30029 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 06:32:45.484694 30029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 06:32:45.484735 30029 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 06:32:45.485311 30029 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 06:32:45.485364 30029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 06:32:45.485491 30051 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Bootstrap starting.
I20260929 06:32:45.487388 30051 log.cc:824] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:45.490474 30051 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:45.490859 30051 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Bootstrap complete.
I20260929 06:32:45.491012 30051 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 06:32:45.491921 30029 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:46153
I20260929 06:32:45.492029 30159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:46153 every 8 connection(s)
I20260929 06:32:45.492272 30029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:32:45.493193 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.492854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:40643)
0929 06:32:45.493022 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.493086 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
W20260929 06:32:45.493399 30160 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:34499 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:34499: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
I20260929 06:32:45.493191 30051 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:45.493579 30051 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff0570dd6fd46dab5595b80fc967061, State: Initialized, Role: FOLLOWER
I20260929 06:32:45.493707 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.493577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:39997)
0929 06:32:45.493660 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.493685 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 06:32:45.493716 30051 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:45.493966 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.493900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:59209)
0929 06:32:45.493927 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.493948 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260929 06:32:45.494011 30051 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 06:32:45.494071 30160 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:32:45.494302 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.494240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:55617)
0929 06:32:45.494271 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.494290 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260929 06:32:45.500434 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30029
I20260929 06:32:45.500685 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:40305
--local_ip_for_outbound_sockets=127.13.15.131
--tserver_master_addrs=127.13.15.190:34499
--webserver_port=41475
--webserver_interface=127.13.15.131
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:32:45.579232 30164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:32:45.579429 30164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:32:45.579452 30164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:32:45.580461 30164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:32:45.580492 30164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:32:45.580516 30164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:32:45.581879 30164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:40305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=41475
--tserver_master_addrs=127.13.15.190:34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:32:45.582074 30164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:32:45.582319 30164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:32:45.587200 30170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.587316 30169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:32:45.587208 30172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:32:45.587406 30164 server_base.cc:1034] running on GCE node
I20260929 06:32:45.587674 30164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:32:45.587908 30164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:32:45.589074 30164 hybrid_clock.cc:648] HybridClock initialized: now 1790663565589056 us; error 30 us; skew 500 ppm
Sep 29 06:32:45 dist-test-slave-sm0p krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:32:45.591636 30164 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:32:45.592047 30164 webserver.cc:460] Webserver started at http://127.13.15.131:41475/ using document root <none> and password file <none>
I20260929 06:32:45.592298 30164 fs_manager.cc:359] Metadata directory not provided
I20260929 06:32:45.592365 30164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:32:45.593654 30164 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:32:45.594283 30179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.594447 30164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:32:45.594544 30164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "c4f247e17e2d48c4a086ac5d62d691db"
format_stamp: "Formatted at 2026-09-29 06:32:04 on dist-test-slave-sm0p"
server_key: "50de3778c37035774b6ead63c703a57f"
server_key_iv: "5ee4868023117ff7c9264b9d6c50b093"
server_key_version: "encryptionkey@0"
I20260929 06:32:45.594884 30164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:32:45.602366 30164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:32:45.604077 30164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:32:45.604233 30164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:32:45.604450 30164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:32:45.604759 30164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:32:45.604809 30164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.604840 30164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:32:45.604864 30164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:32:45.610353 30164 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40305
I20260929 06:32:45.610399 30292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40305 every 8 connection(s)
I20260929 06:32:45.610704 30164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:32:45.611749 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.611335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:48525)
0929 06:32:45.611490 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.611593 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260929 06:32:45.612123 30293 heartbeater.cc:644] Failed to heartbeat to 127.13.15.190:34499 (0 consecutive failures): Network error: Failed to ping master at 127.13.15.190:34499: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
I20260929 06:32:45.612488 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.612383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:44547)
0929 06:32:45.612433 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.612470 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 06:32:45.612746 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:45.612664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:56341)
0929 06:32:45.612707 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:45.612728 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:32:45.612867 30293 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 06:32:45.615163 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30164
I20260929 06:32:46.438421 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.438170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:45695)
0929 06:32:46.438324 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.438359 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260929 06:32:46.494837 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.494636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:57003)
0929 06:32:46.494762 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.494795 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260929 06:32:46.613305 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.613127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:50417)
0929 06:32:46.613227 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.613256 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260929 06:32:46.698379 30300 raft_consensus.cc:482] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:32:46.698498 30300 raft_consensus.cc:504] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:46.698894 30300 leader_election.cc:288] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305), 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153)
Sep 29 06:32:46 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.131@KRBTEST.COM
Sep 29 06:32:46 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.130@KRBTEST.COM
I20260929 06:32:46.709026 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.699121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53371 (local address 127.13.15.131:40305)
0929 06:32:46.699373 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:46.699382 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:46.699406 (+ 24us) server_negotiation.cc:407] Connection header received
0929 06:32:46.699981 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:46.699984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:46.700033 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:46.700163 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:46.700601 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.701529 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.702650 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.703066 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.703194 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:46.705492 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:46.705496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:46.705496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:46.705505 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:46.707377 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:46.707814 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:46.707817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:46.707817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:46.707914 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:46.708175 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:46.708176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:46.708176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:46.708422 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:46.708509 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:46.708791 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 06:32:46.708919 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":60,"threads_started":1}
I20260929 06:32:46.709026 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.699010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40305 (local address 127.13.15.129:53371)
0929 06:32:46.699061 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.699071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:46.699123 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:46.700156 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:46.700156 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:46.700329 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:46.700474 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:46.700481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.701630 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.701631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:46.702563 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:46.702566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.703162 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.703163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:46.703286 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:46.703861 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:46.703869 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:46.705386 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:46.707485 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:46.707492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:46.707495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:46.707714 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:46.707987 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:46.707988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:46.707988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:46.708096 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:46.708502 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:46.708509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:46.708635 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 06:32:46.708740 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 06:32:46.708913 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 06:32:46.709156 30301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.699042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:46153 (local address 127.13.15.129:35561)
0929 06:32:46.699188 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.699198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:46.699252 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:46.699652 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:46.699655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:46.700036 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:46.700584 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:46.700587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.701986 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.701986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:46.702570 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:46.702572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.703159 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.703160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:46.703281 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:46.703920 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:46.703925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:46.705831 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:46.707840 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:46.707841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:46.707842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:46.708017 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:46.708275 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:46.708276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:46.708276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:46.708349 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:46.708712 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:46.708713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:46.708755 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 06:32:46.708835 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:32:46.709003 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":61,"thread_start_us":62,"threads_started":1}
I20260929 06:32:46.709728 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.699174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:35561 (local address 127.13.15.130:46153)
0929 06:32:46.699314 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:46.699323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:46.699342 (+ 19us) server_negotiation.cc:407] Connection header received
0929 06:32:46.699441 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:46.699442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:46.699484 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:46.699585 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:46.700676 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.701879 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.702676 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:46.703065 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:46.703194 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:46.705942 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:46.705954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:46.705954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:46.705966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:46.707736 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:46.708112 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:46.708114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:46.708115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:46.708204 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:46.708415 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:46.708416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:46.708417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:46.708633 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:46.708716 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:46.708893 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 06:32:46.709040 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
W20260929 06:32:46.709885 29912 leader_election.cc:341] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305): Not found: Tablet not found: 88a2f6944411431f8c683266b4c5469e
I20260929 06:32:46.710194 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "62e8196632914032a769a4de48e8e5ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9ff0570dd6fd46dab5595b80fc967061" is_pre_election: true
I20260929 06:32:46.710367 30114 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e8196632914032a769a4de48e8e5ad in term 1.
I20260929 06:32:46.710650 29911 leader_election.cc:302] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62e8196632914032a769a4de48e8e5ad, 9ff0570dd6fd46dab5595b80fc967061; no voters: c4f247e17e2d48c4a086ac5d62d691db
I20260929 06:32:46.710777 30300 raft_consensus.cc:2781] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 06:32:46.710839 30300 raft_consensus.cc:482] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:32:46.710863 30300 raft_consensus.cc:3037] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:32:46.711987 30300 raft_consensus.cc:504] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:46.712146 30300 leader_election.cc:288] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 election: Requested vote from peers c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305), 9ff0570dd6fd46dab5595b80fc967061 (127.13.15.130:46153)
I20260929 06:32:46.712347 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a2f6944411431f8c683266b4c5469e" candidate_uuid: "62e8196632914032a769a4de48e8e5ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9ff0570dd6fd46dab5595b80fc967061"
W20260929 06:32:46.712412 29912 leader_election.cc:341] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305): Not found: Tablet not found: 88a2f6944411431f8c683266b4c5469e
I20260929 06:32:46.712436 30114 raft_consensus.cc:3037] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 1 FOLLOWER]: Advancing to term 2
I20260929 06:32:46.713613 30114 raft_consensus.cc:2454] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e8196632914032a769a4de48e8e5ad in term 2.
I20260929 06:32:46.713816 29911 leader_election.cc:302] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62e8196632914032a769a4de48e8e5ad, 9ff0570dd6fd46dab5595b80fc967061; no voters: c4f247e17e2d48c4a086ac5d62d691db
I20260929 06:32:46.713915 30300 raft_consensus.cc:2781] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 2 FOLLOWER]: Leader election won for term 2
I20260929 06:32:46.714053 30300 raft_consensus.cc:686] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [term 2 LEADER]: Becoming Leader. State: Replica: 62e8196632914032a769a4de48e8e5ad, State: Running, Role: LEADER
I20260929 06:32:46.714172 30300 consensus_queue.cc:237] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:46.714457 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:46.714269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:52667)
0929 06:32:46.714393 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:46.714422 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260929 06:32:47.166280 29912 consensus_peers.cc:487] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad -> Peer c4f247e17e2d48c4a086ac5d62d691db (127.13.15.131:40305): Couldn't send request to peer c4f247e17e2d48c4a086ac5d62d691db. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88a2f6944411431f8c683266b4c5469e. This is attempt 1: this message will repeat every 5th retry.
I20260929 06:32:47.242612 30114 raft_consensus.cc:1261] T 88a2f6944411431f8c683266b4c5469e P 9ff0570dd6fd46dab5595b80fc967061 [term 2 FOLLOWER]: Refusing update from remote peer 62e8196632914032a769a4de48e8e5ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260929 06:32:47.243002 30300 consensus_queue.cc:1035] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260929 06:32:47.243988 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.243780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:55179)
0929 06:32:47.243875 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.243938 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
I20260929 06:32:47.244961 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.244778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:44487)
0929 06:32:47.244859 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.244915 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":44,"threads_started":1}
I20260929 06:32:47.244946 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.244850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:57671)
0929 06:32:47.244903 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.244927 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 06:32:47.548344 30313 ts_tablet_manager.cc:927] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Initiating tablet copy from peer 62e8196632914032a769a4de48e8e5ad (127.13.15.129:38511)
I20260929 06:32:47.548583 30313 tablet_copy_client.cc:316] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: tablet copy: Beginning tablet copy session from remote peer at address 127.13.15.129:38511
Sep 29 06:32:47 dist-test-slave-sm0p krb5kdc[28722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.129@KRBTEST.COM
I20260929 06:32:47.562224 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.548965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:40681 (local address 127.13.15.129:38511)
0929 06:32:47.549140 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 06:32:47.549148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:32:47.549160 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:32:47.549185 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:32:47.549187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:32:47.549204 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:32:47.549290 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:32:47.549720 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:47.550457 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:47.551195 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:32:47.551489 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:32:47.551541 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:47.559600 (+ 8059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:32:47.559609 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:32:47.559609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:32:47.559616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:32:47.561077 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:47.561458 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:47.561461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:47.561461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:47.561516 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:32:47.561680 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:32:47.561681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:32:47.561681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:32:47.561817 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:32:47.561877 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:32:47.562042 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 06:32:47.562103 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260929 06:32:47.562230 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.548846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:38511 (local address 127.13.15.131:40681)
0929 06:32:47.549046 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.549062 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 06:32:47.549123 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:32:47.549298 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:32:47.549301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:32:47.549583 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:32:47.549646 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:47.549648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:47.550523 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:47.550524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:47.551123 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:32:47.551125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:32:47.551552 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:32:47.551553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:32:47.551613 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:32:47.557899 (+ 6286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:32:47.557904 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:32:47.559500 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:32:47.561163 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:47.561170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:47.561172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:47.561385 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:47.561574 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:32:47.561575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:32:47.561575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:32:47.561629 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:32:47.561874 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:32:47.561880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:32:47.561926 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 06:32:47.561992 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 06:32:47.562090 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":141,"threads_started":1}
I20260929 06:32:47.562708 29999 tablet_copy_service.cc:140] P 62e8196632914032a769a4de48e8e5ad: Received BeginTabletCopySession request for tablet 88a2f6944411431f8c683266b4c5469e from peer c4f247e17e2d48c4a086ac5d62d691db ({username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:40681)
I20260929 06:32:47.562778 29999 tablet_copy_service.cc:161] P 62e8196632914032a769a4de48e8e5ad: Beginning new tablet copy session on tablet 88a2f6944411431f8c683266b4c5469e from peer c4f247e17e2d48c4a086ac5d62d691db at {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:40681: session id = c4f247e17e2d48c4a086ac5d62d691db-88a2f6944411431f8c683266b4c5469e
I20260929 06:32:47.563302 29999 tablet_copy_source_session.cc:205] T 88a2f6944411431f8c683266b4c5469e P 62e8196632914032a769a4de48e8e5ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 06:32:47.564287 30313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a2f6944411431f8c683266b4c5469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:32:47.566973 30313 tablet_copy_client.cc:791] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: tablet copy: Starting download of 0 data blocks...
I20260929 06:32:47.567134 30313 tablet_copy_client.cc:655] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: tablet copy: Starting download of 1 WAL segments...
I20260929 06:32:47.567912 30313 tablet_copy_client.cc:531] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 06:32:47.568611 30313 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Bootstrap starting.
I20260929 06:32:47.569991 30313 log.cc:824] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Log is configured to *not* fsync() on all Append() calls
I20260929 06:32:47.572355 30313 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 06:32:47.572631 30313 tablet_bootstrap.cc:492] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Bootstrap complete.
I20260929 06:32:47.572760 30313 ts_tablet_manager.cc:1397] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 06:32:47.573693 30313 raft_consensus.cc:348] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:47.573828 30313 raft_consensus.cc:729] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f247e17e2d48c4a086ac5d62d691db, State: Initialized, Role: FOLLOWER
I20260929 06:32:47.573961 30313 consensus_queue.cc:260] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f247e17e2d48c4a086ac5d62d691db" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40305 } } peers { permanent_uuid: "9ff0570dd6fd46dab5595b80fc967061" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 46153 } } peers { permanent_uuid: "62e8196632914032a769a4de48e8e5ad" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 38511 } }
I20260929 06:32:47.574244 30313 ts_tablet_manager.cc:1428] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20260929 06:32:47.574529 29999 tablet_copy_service.cc:342] P 62e8196632914032a769a4de48e8e5ad: Request end of tablet copy session c4f247e17e2d48c4a086ac5d62d691db-88a2f6944411431f8c683266b4c5469e received from {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:40681
I20260929 06:32:47.574561 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.574455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:40059)
0929 06:32:47.574503 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.574530 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 06:32:47.574597 29999 tablet_copy_service.cc:433] P 62e8196632914032a769a4de48e8e5ad: ending tablet copy session c4f247e17e2d48c4a086ac5d62d691db-88a2f6944411431f8c683266b4c5469e on tablet 88a2f6944411431f8c683266b4c5469e with peer c4f247e17e2d48c4a086ac5d62d691db
I20260929 06:32:47.574904 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.574798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:34619)
0929 06:32:47.574834 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.574882 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 06:32:47.964963 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.964773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:34661)
0929 06:32:47.964884 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.964918 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260929 06:32:47.964967 30247 raft_consensus.cc:1203] T 88a2f6944411431f8c683266b4c5469e P c4f247e17e2d48c4a086ac5d62d691db [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260929 06:32:47.965480 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:47.965390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:55811)
0929 06:32:47.965425 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:47.965453 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 06:32:48.245484 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:48.245306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:47119)
0929 06:32:48.245407 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:48.245441 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":47,"threads_started":1}
I20260929 06:32:48.965794 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:48.965567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:49207)
0929 06:32:48.965709 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:48.965749 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260929 06:32:48.966324 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:48.966094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:51981)
0929 06:32:48.966240 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:48.966274 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260929 06:32:49.246110 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:49.245900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:52783)
0929 06:32:49.246032 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:49.246065 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260929 06:32:49.966552 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:49.966369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:51667)
0929 06:32:49.966481 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:49.966507 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260929 06:32:49.966926 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:49.966723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:51385)
0929 06:32:49.966857 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:49.966889 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260929 06:32:50.246742 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:50.246557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:59801)
0929 06:32:50.246663 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:50.246694 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260929 06:32:50.967219 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:50.967005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:34233)
0929 06:32:50.967137 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:50.967177 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260929 06:32:50.967626 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:50.967406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:36591)
0929 06:32:50.967547 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:50.967584 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260929 06:32:50.967669 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:50.967546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:54293)
0929 06:32:50.967616 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:50.967645 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 06:32:51.247392 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:51.247203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:34253)
0929 06:32:51.247314 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:51.247349 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260929 06:32:51.968405 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:51.968196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:35865)
0929 06:32:51.968323 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:51.968361 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":96,"threads_started":1}
I20260929 06:32:51.968411 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:51.968217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:42565)
0929 06:32:51.968334 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:51.968369 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260929 06:32:52.248214 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:52.247947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:55307)
0929 06:32:52.248122 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:52.248166 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260929 06:32:52.969065 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:52.968831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:36865)
0929 06:32:52.968987 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:52.969023 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260929 06:32:52.969065 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:52.968846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:39059)
0929 06:32:52.968991 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:52.969021 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260929 06:32:53.248852 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:53.248639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:39815)
0929 06:32:53.248772 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:53.248806 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260929 06:32:53.969781 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:53.969589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:60483)
0929 06:32:53.969707 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:53.969735 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260929 06:32:53.969807 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:53.969585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:41137)
0929 06:32:53.969729 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:53.969764 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260929 06:32:54.249523 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:54.249301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:54331)
0929 06:32:54.249444 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:54.249477 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260929 06:32:54.970456 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:54.970216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:35631)
0929 06:32:54.970360 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:54.970399 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260929 06:32:54.970604 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:54.970369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:53117)
0929 06:32:54.970501 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:54.970536 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260929 06:32:55.250412 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:55.250215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:41661)
0929 06:32:55.250333 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:55.250370 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260929 06:32:55.971168 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:55.970952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:47585)
0929 06:32:55.971085 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:55.971124 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260929 06:32:55.971293 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:55.971051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:49551)
0929 06:32:55.971184 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:55.971227 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260929 06:32:56.251050 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:56.250853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:41291)
0929 06:32:56.250972 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:56.251007 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260929 06:32:56.971822 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:56.971616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:48007)
0929 06:32:56.971746 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:56.971776 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260929 06:32:56.971961 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:56.971717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:54235)
0929 06:32:56.971834 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:56.971898 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260929 06:32:57.251731 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:57.251533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:40161)
0929 06:32:57.251654 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:57.251687 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260929 06:32:57.972568 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:57.972283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:39561)
0929 06:32:57.972461 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:57.972505 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260929 06:32:57.972720 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:57.972493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:58809)
0929 06:32:57.972612 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:57.972668 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260929 06:32:58.252409 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:58.252187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:34181)
0929 06:32:58.252321 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:58.252356 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260929 06:32:58.973219 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:58.972999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:49907)
0929 06:32:58.973139 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:58.973176 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260929 06:32:58.973392 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:58.973140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:48555)
0929 06:32:58.973266 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:58.973334 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260929 06:32:59.253083 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:59.252833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:39679)
0929 06:32:59.252983 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:59.253024 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":59,"threads_started":1}
I20260929 06:32:59.973869 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:59.973647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.129:49145)
0929 06:32:59.973795 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:59.973826 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260929 06:32:59.973991 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:32:59.973794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.131:35867)
0929 06:32:59.973900 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 06:32:59.973925 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":44,"threads_started":1}
I20260929 06:33:00.253876 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:00.253637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34499 (local address 127.13.15.130:54169)
0929 06:33:00.253801 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:00.253831 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.190:34499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260929 06:33:00.623054 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 29861
I20260929 06:33:00.627731 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30029
I20260929 06:33:00.632045 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30164
2026-09-29T06:33:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58107 ms)
[----------] 30 tests from SecurityITest (256924 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:00 dist-test-slave-sm0p krb5kdc[30361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:00 dist-test-slave-sm0p krb5kdc[30361](info): set up 2 sockets
Sep 29 06:33:00 dist-test-slave-sm0p krb5kdc[30361](info): commencing operation
krb5kdc: starting...
W20260929 06:33:01.653957 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:01 dist-test-slave-sm0p krb5kdc[30361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:01.768301 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44263
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:01.849629 30374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:01.849864 30374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:01.849901 30374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:01.851081 30374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:01.851143 30374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:01.851167 30374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:01.851194 30374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:01.851221 30374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:01.852792 30374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:01.853056 30374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:01.853229 30374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:01.858094 30374 server_base.cc:1034] running on GCE node
W20260929 06:33:01.858175 30380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:01.858168 30379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:01.858300 30382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:01.858556 30374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:01.858798 30374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:01.859937 30374 hybrid_clock.cc:648] HybridClock initialized: now 1790663581859914 us; error 40 us; skew 500 ppm
Sep 29 06:33:01 dist-test-slave-sm0p krb5kdc[30361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663581, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:01.862323 30374 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:01.862764 30374 webserver.cc:460] Webserver started at http://127.13.15.190:45769/ using document root <none> and password file <none>
I20260929 06:33:01.862979 30374 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:01.863023 30374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:01.863162 30374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:01.864183 30374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "54782b71f30b43f9b5e4165810bb08cb"
format_stamp: "Formatted at 2026-09-29 06:33:01 on dist-test-slave-sm0p"
server_key: "02d304302c6b069ea5513f26720fb68c"
server_key_iv: "cd16119ca6395e7f9aada9debc7b1ecb"
server_key_version: "encryptionkey@0"
I20260929 06:33:01.864542 30374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "54782b71f30b43f9b5e4165810bb08cb"
format_stamp: "Formatted at 2026-09-29 06:33:01 on dist-test-slave-sm0p"
server_key: "02d304302c6b069ea5513f26720fb68c"
server_key_iv: "cd16119ca6395e7f9aada9debc7b1ecb"
server_key_version: "encryptionkey@0"
I20260929 06:33:01.865813 30374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:01.866588 30389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:01.866737 30374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:01.866823 30374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "54782b71f30b43f9b5e4165810bb08cb"
format_stamp: "Formatted at 2026-09-29 06:33:01 on dist-test-slave-sm0p"
server_key: "02d304302c6b069ea5513f26720fb68c"
server_key_iv: "cd16119ca6395e7f9aada9debc7b1ecb"
server_key_version: "encryptionkey@0"
I20260929 06:33:01.866878 30374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:01.882726 30374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:01.884490 30374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:01.884598 30374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:01.888299 30374 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44263
I20260929 06:33:01.888355 30441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44263 every 8 connection(s)
I20260929 06:33:01.888690 30374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:01.889552 30442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:01.891759 30442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: Bootstrap starting.
I20260929 06:33:01.892277 30442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:01.892508 30442 log.cc:824] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:01.892777 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30374
I20260929 06:33:01.892899 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:01.893133 13374 external_mini_cluster.cc:1428] Setting key 28f92e1a06412cb48f7b150c58259ca6
I20260929 06:33:01.893363 30442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: No bootstrap required, opened a new log
I20260929 06:33:01.895525 30442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } }
I20260929 06:33:01.895702 30442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:01.895747 30442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54782b71f30b43f9b5e4165810bb08cb, State: Initialized, Role: FOLLOWER
I20260929 06:33:01.895906 30442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } }
I20260929 06:33:01.895982 30442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:01.896011 30442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:01.896054 30442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:01.896697 30442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } }
I20260929 06:33:01.896812 30442 leader_election.cc:302] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54782b71f30b43f9b5e4165810bb08cb; no voters:
I20260929 06:33:01.896950 30442 leader_election.cc:288] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:01.897002 30447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:01.897140 30447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [term 1 LEADER]: Becoming Leader. State: Replica: 54782b71f30b43f9b5e4165810bb08cb, State: Running, Role: LEADER
I20260929 06:33:01.897161 30442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:01.898420 30447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } }
I20260929 06:33:01.898954 30448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54782b71f30b43f9b5e4165810bb08cb. Latest consensus state: current_term: 1 leader_uuid: "54782b71f30b43f9b5e4165810bb08cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } } }
I20260929 06:33:01.899048 30449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54782b71f30b43f9b5e4165810bb08cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54782b71f30b43f9b5e4165810bb08cb" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44263 } } }
I20260929 06:33:01.899161 30448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:01.899165 30449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:01.899366 30463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:01.899883 30463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:33:01 dist-test-slave-sm0p krb5kdc[30361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:01.902004 30463 catalog_manager.cc:1347] Generated new cluster ID: f3b88a544770485699290680f63d3278
I20260929 06:33:01.902057 30463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:01.905607 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:01.893623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.13.15.190:44263)
0929 06:33:01.893817 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:01.893825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:01.893857 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:33:01.894727 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:01.894735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:01.894797 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:01.894950 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:01.895508 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:01.897143 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:01.898290 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:01.898898 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:01.900794 (+ 1896us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:01.901910 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:01.901913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:01.901914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:01.901925 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:01.903843 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:01.904259 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:01.904262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:01.904263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:01.904363 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:01.904570 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:01.904572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:01.904572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:01.904899 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:01.905054 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:01.905160 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 06:33:01.905317 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260929 06:33:01.911072 30463 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:01.911579 30463 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:01.916110 30463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: Generated new TSK 0
I20260929 06:33:01.916289 30463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:33:01.944252 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:02.024341 30468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.024535 30468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.024559 30468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.025609 30468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.025643 30468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.025669 30468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:02.027015 30468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.027194 30468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.027359 30468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:02.032305 30476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.032318 30473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.032312 30474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.032492 30468 server_base.cc:1034] running on GCE node
I20260929 06:33:02.032744 30468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.032948 30468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.034096 30468 hybrid_clock.cc:648] HybridClock initialized: now 1790663582034089 us; error 29 us; skew 500 ppm
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:02.036566 30468 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:33:02.036991 30468 webserver.cc:460] Webserver started at http://127.13.15.129:39973/ using document root <none> and password file <none>
I20260929 06:33:02.037202 30468 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.037266 30468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.037415 30468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.038458 30468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "ca694749064b4ce38acb8ee0ac900d0e"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "8b5b0a1b412451b3d16075e911823d5d"
server_key_iv: "e5a73c8c50f939779126f30bb77a0971"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.038795 30468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "ca694749064b4ce38acb8ee0ac900d0e"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "8b5b0a1b412451b3d16075e911823d5d"
server_key_iv: "e5a73c8c50f939779126f30bb77a0971"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.040115 30468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:02.040814 30483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.040980 30468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:02.041088 30468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "ca694749064b4ce38acb8ee0ac900d0e"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "8b5b0a1b412451b3d16075e911823d5d"
server_key_iv: "e5a73c8c50f939779126f30bb77a0971"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.041167 30468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.048108 30468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.050354 30468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.050503 30468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.050722 30468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:02.051012 30468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:02.051067 30468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.051106 30468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:02.051136 30468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.056921 30468 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:44493
I20260929 06:33:02.056988 30596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:44493 every 8 connection(s)
I20260929 06:33:02.057299 30468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:02.058802 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30468
I20260929 06:33:02.058919 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:02.059161 13374 external_mini_cluster.cc:1428] Setting key a17120316b0e7b99fb4a5fc33ba81777
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:33:02.067687 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.058053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56647 (local address 127.13.15.190:44263)
0929 06:33:02.058104 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.058106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.058372 (+ 266us) server_negotiation.cc:407] Connection header received
0929 06:33:02.058983 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.058984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.058996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.059061 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.060336 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.060811 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.061926 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.062335 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.062412 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.064576 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.064578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.064578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:02.064586 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.066288 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.066686 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.066687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.066688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.066752 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.066993 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.066994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.066995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.067142 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.067219 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.067465 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.067576 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:33:02.067794 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.058037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44263 (local address 127.13.15.129:56647)
0929 06:33:02.058242 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.058282 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.058878 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.059146 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.059150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.059950 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:02.060221 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.060227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.060902 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.060903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.061836 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.061838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.062434 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.062435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.062538 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.063069 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:02.063077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:02.064482 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:02.066375 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.066381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.066383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.066607 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.066822 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.066823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.066823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.066921 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.067210 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:02.067214 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:02.067334 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.067418 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.067576 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260929 06:33:02.068321 30597 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44263
I20260929 06:33:02.068405 30597 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:02.068593 30597 heartbeater.cc:511] Master 127.13.15.190:44263 requested a full tablet report, sending...
I20260929 06:33:02.069096 30406 ts_manager.cc:194] Registered new tserver with Master: ca694749064b4ce38acb8ee0ac900d0e (127.13.15.129:44493)
I20260929 06:33:02.069806 30406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:56647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:33:02.084529 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:02.164144 30602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.164336 30602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.164359 30602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.165400 30602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.165432 30602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.165457 30602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:02.166800 30602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.166973 30602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.167131 30602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:02.172113 30602 server_base.cc:1034] running on GCE node
W20260929 06:33:02.172088 30610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.172096 30608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.172092 30607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.172474 30602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.172719 30602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.173885 30602 hybrid_clock.cc:648] HybridClock initialized: now 1790663582173874 us; error 43 us; skew 500 ppm
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:02.176327 30602 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:33:02.176822 30602 webserver.cc:460] Webserver started at http://127.13.15.130:43555/ using document root <none> and password file <none>
I20260929 06:33:02.177064 30602 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.177130 30602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.177261 30602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.178351 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "d4a01ec0adfc455091f33244f15b36a5"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "12cf2810934a3f5f49d933dd4153a2e7"
server_key_iv: "f1f039082ada48f72559890c494e48a3"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.178720 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "d4a01ec0adfc455091f33244f15b36a5"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "12cf2810934a3f5f49d933dd4153a2e7"
server_key_iv: "f1f039082ada48f72559890c494e48a3"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.179983 30602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:02.180694 30617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.180864 30602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:02.180960 30602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "d4a01ec0adfc455091f33244f15b36a5"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "12cf2810934a3f5f49d933dd4153a2e7"
server_key_iv: "f1f039082ada48f72559890c494e48a3"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.181039 30602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.189814 30602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.191715 30602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.191872 30602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.192101 30602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:02.192406 30602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:02.192461 30602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.192502 30602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:02.192530 30602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.198758 30602 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:39647
I20260929 06:33:02.198832 30730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:39647 every 8 connection(s)
I20260929 06:33:02.199136 30602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:02.207451 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.199891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:51015 (local address 127.13.15.190:44263)
0929 06:33:02.199938 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.199941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.200167 (+ 226us) server_negotiation.cc:407] Connection header received
0929 06:33:02.200602 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.200603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.200613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.200659 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.201528 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.202016 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.202990 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.203171 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.203224 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.205196 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.205197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.205197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:02.205202 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.206392 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.206731 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.206731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.206732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.206778 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.206974 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.206975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.206975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.207065 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.207112 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.207324 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.207391 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:02.207592 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.199887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44263 (local address 127.13.15.130:51015)
0929 06:33:02.200051 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.200084 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.200524 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.200717 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.200721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.201273 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:02.201455 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.201460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.202091 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.202092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.202921 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.202923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.203242 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.203243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.203338 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.203854 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:02.203862 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:02.205122 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:02.206457 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.206463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.206464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.206668 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.206833 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.206834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.206834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.206923 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.207117 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:02.207121 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:02.207221 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.207300 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.207407 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260929 06:33:02.208074 30731 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44263
I20260929 06:33:02.208158 30731 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:02.208287 30731 heartbeater.cc:511] Master 127.13.15.190:44263 requested a full tablet report, sending...
I20260929 06:33:02.208624 30406 ts_manager.cc:194] Registered new tserver with Master: d4a01ec0adfc455091f33244f15b36a5 (127.13.15.130:39647)
I20260929 06:33:02.208993 30406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:51015
I20260929 06:33:02.209029 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30602
I20260929 06:33:02.209141 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:02.209407 13374 external_mini_cluster.cc:1428] Setting key 38e5023ab960157563f319f76b7988cd
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:33:02.233100 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:02.313284 30736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.313478 30736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.313514 30736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.314608 30736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.314646 30736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.314671 30736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:02.316033 30736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.316212 30736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.316385 30736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:02.321200 30741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.321233 30742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.321399 30736 server_base.cc:1034] running on GCE node
W20260929 06:33:02.321233 30744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.321686 30736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.321934 30736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.323099 30736 hybrid_clock.cc:648] HybridClock initialized: now 1790663582323156 us; error 98 us; skew 500 ppm
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:02.325687 30736 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:33:02.326179 30736 webserver.cc:460] Webserver started at http://127.13.15.131:44783/ using document root <none> and password file <none>
I20260929 06:33:02.326390 30736 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.326431 30736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.326517 30736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.327486 30736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "0947463b223941e2bb5e1528ffba2886"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "4284a6aeeefd473f652f1b0f3909acd0"
server_key_iv: "c88d07f79e4afb0ee40b85c8c7946b73"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.327786 30736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "0947463b223941e2bb5e1528ffba2886"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "4284a6aeeefd473f652f1b0f3909acd0"
server_key_iv: "c88d07f79e4afb0ee40b85c8c7946b73"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.328965 30736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:02.329617 30751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.329785 30736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:02.329866 30736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "0947463b223941e2bb5e1528ffba2886"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "4284a6aeeefd473f652f1b0f3909acd0"
server_key_iv: "c88d07f79e4afb0ee40b85c8c7946b73"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.329921 30736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.353487 30736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.355453 30736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.355580 30736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.355772 30736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:02.356026 30736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:02.356058 30736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.356081 30736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:02.356092 30736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.361460 30736 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:41829
I20260929 06:33:02.361544 30864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:41829 every 8 connection(s)
I20260929 06:33:02.361833 30736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
Sep 29 06:33:02 dist-test-slave-sm0p krb5kdc[30361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663582, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:02.367609 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30736
I20260929 06:33:02.367738 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:02.367980 13374 external_mini_cluster.cc:1428] Setting key 68ae8c84c4d76d154f053125132386fa
I20260929 06:33:02.370776 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.362615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:45189 (local address 127.13.15.190:44263)
0929 06:33:02.362682 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.362685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.362871 (+ 186us) server_negotiation.cc:407] Connection header received
0929 06:33:02.363331 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.363332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.363342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.363403 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.364273 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.364763 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.365683 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.365866 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.365914 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.367947 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.367949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.367949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:02.367957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.369455 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.369875 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.369877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.369877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.369934 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.370241 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.370242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.370243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.370374 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.370426 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.370654 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.370715 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:33:02.370932 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.362595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:44263 (local address 127.13.15.131:45189)
0929 06:33:02.362753 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.362791 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.363248 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.363471 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.363474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.364024 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:02.364203 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.364208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.364834 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.364834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.365615 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.365617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.365934 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.365935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.366031 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.366476 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:02.366483 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:02.367854 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:02.369556 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.369563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.369565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.369781 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.370018 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:02.370018 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:02.370019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:02.370123 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:02.370426 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:02.370431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:02.370540 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.370623 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.370737 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260929 06:33:02.371382 30865 heartbeater.cc:347] Connected to a master server at 127.13.15.190:44263
I20260929 06:33:02.371470 30865 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:02.371605 30865 heartbeater.cc:511] Master 127.13.15.190:44263 requested a full tablet report, sending...
I20260929 06:33:02.371965 30406 ts_manager.cc:194] Registered new tserver with Master: 0947463b223941e2bb5e1528ffba2886 (127.13.15.131:41829)
I20260929 06:33:02.372323 30406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:45189
I20260929 06:33:02.379770 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:02.386857 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.381128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.13.15.190:44263)
0929 06:33:02.381186 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.381188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.381295 (+ 107us) server_negotiation.cc:407] Connection header received
0929 06:33:02.381333 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.381333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.381342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.381390 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.381928 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.382482 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.383171 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.383348 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.383406 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.384636 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.384637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.384638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:02.384642 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.385893 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.386254 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.386255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.386256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.386306 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:02.386439 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:02.386439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:02.386440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:02.386568 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.386639 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.386724 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.386793 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 06:33:02.388460 30406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:02.389954 30406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:02.396363 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.13.15.129:44493)
0929 06:33:02.392612 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.392618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.392696 (+ 78us) server_negotiation.cc:407] Connection header received
0929 06:33:02.392784 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.392786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.392804 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.392887 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:02.393084 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.394231 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.395447 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.395949 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396040 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396102 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.396217 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.396259 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260929 06:33:02.396399 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44493 (local address 127.0.0.1:41668)
0929 06:33:02.392643 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.392652 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.392708 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.392964 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.392965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.392966 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:02.393003 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.393004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.394339 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.394341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.395369 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.395371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.396038 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.396115 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396129 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.396202 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.396259 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":90,"threads_started":1}
I20260929 06:33:02.396428 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:41829 (local address 127.0.0.1:60994)
0929 06:33:02.392431 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.392441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.392498 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.392950 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.392951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.392953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:02.393004 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.393004 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.394398 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.394400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.395325 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.395327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.396174 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.396235 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396238 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.396265 (+ 27us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.396294 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":68,"threads_started":1}
I20260929 06:33:02.396411 30873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39647 (local address 127.0.0.1:41500)
0929 06:33:02.392194 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.392233 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.392295 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.392875 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.392878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.392880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:02.392946 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.392947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.393889 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.393890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.395188 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.395191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.396126 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.396193 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396197 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.396227 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.396276 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260929 06:33:02.396615 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.13.15.131:41829)
0929 06:33:02.392638 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.392644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.392654 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:02.392696 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.392698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.392715 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.392840 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:02.393176 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.394276 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.395415 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.396095 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396401 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396464 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.396491 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.396538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":110,"threads_started":1}
I20260929 06:33:02.396834 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.392458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.13.15.130:39647)
0929 06:33:02.392641 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.392645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.392654 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:02.392696 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.392698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.392714 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.393025 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:02.393106 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.393785 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.395276 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.396021 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.396575 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.396638 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.396664 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.396707 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260929 06:33:02.397482 30665 tablet_service.cc:1467] Processing CreateTablet for tablet bf2390d67959403689f7f5409536f3cc (DEFAULT_TABLE table=test-table [id=e4124efd5f36463690c7e2753b668294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:02.397632 30799 tablet_service.cc:1467] Processing CreateTablet for tablet bf2390d67959403689f7f5409536f3cc (DEFAULT_TABLE table=test-table [id=e4124efd5f36463690c7e2753b668294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:02.397806 30665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2390d67959403689f7f5409536f3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:02.397977 30799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2390d67959403689f7f5409536f3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:02.400609 30879 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Bootstrap starting.
I20260929 06:33:02.400622 30880 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Bootstrap starting.
I20260929 06:33:02.401073 30531 tablet_service.cc:1467] Processing CreateTablet for tablet bf2390d67959403689f7f5409536f3cc (DEFAULT_TABLE table=test-table [id=e4124efd5f36463690c7e2753b668294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:02.401369 30879 tablet_bootstrap.cc:654] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:02.401413 30531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2390d67959403689f7f5409536f3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:02.401477 30880 tablet_bootstrap.cc:654] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:02.401901 30879 log.cc:824] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:02.402070 30880 log.cc:824] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:02.402683 30879 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: No bootstrap required, opened a new log
I20260929 06:33:02.402765 30879 ts_tablet_manager.cc:1397] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:02.404053 30879 raft_consensus.cc:348] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.404214 30879 raft_consensus.cc:374] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:02.404203 30883 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: Bootstrap starting.
I20260929 06:33:02.404264 30879 raft_consensus.cc:729] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0947463b223941e2bb5e1528ffba2886, State: Initialized, Role: FOLLOWER
I20260929 06:33:02.404387 30879 consensus_queue.cc:260] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.404651 30879 ts_tablet_manager.cc:1428] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 06:33:02.404803 30865 heartbeater.cc:503] Master 127.13.15.190:44263 was elected leader, sending a full tablet report...
I20260929 06:33:02.404873 30883 tablet_bootstrap.cc:654] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:02.405342 30880 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: No bootstrap required, opened a new log
I20260929 06:33:02.405414 30880 ts_tablet_manager.cc:1397] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260929 06:33:02.405416 30883 log.cc:824] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:02.406353 30883 tablet_bootstrap.cc:492] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: No bootstrap required, opened a new log
I20260929 06:33:02.406424 30883 ts_tablet_manager.cc:1397] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:02.407385 30880 raft_consensus.cc:348] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.407552 30880 raft_consensus.cc:374] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:02.407588 30880 raft_consensus.cc:729] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a01ec0adfc455091f33244f15b36a5, State: Initialized, Role: FOLLOWER
I20260929 06:33:02.407691 30880 consensus_queue.cc:260] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.407968 30880 ts_tablet_manager.cc:1428] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 06:33:02.408123 30731 heartbeater.cc:503] Master 127.13.15.190:44263 was elected leader, sending a full tablet report...
I20260929 06:33:02.408360 30883 raft_consensus.cc:348] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.408636 30883 raft_consensus.cc:374] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:02.408684 30883 raft_consensus.cc:729] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca694749064b4ce38acb8ee0ac900d0e, State: Initialized, Role: FOLLOWER
I20260929 06:33:02.408780 30883 consensus_queue.cc:260] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.409114 30597 heartbeater.cc:503] Master 127.13.15.190:44263 was elected leader, sending a full tablet report...
I20260929 06:33:02.409045 30883 ts_tablet_manager.cc:1428] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 06:33:02.449697 30732 tablet.cc:2369] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:02.453313 30884 raft_consensus.cc:482] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:02.453409 30884 raft_consensus.cc:504] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.453800 30884 leader_election.cc:288] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4a01ec0adfc455091f33244f15b36a5 (127.13.15.130:39647), ca694749064b4ce38acb8ee0ac900d0e (127.13.15.129:44493)
I20260929 06:33:02.456773 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.454104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:49325 (local address 127.13.15.130:39647)
0929 06:33:02.454204 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.454206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.454215 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:02.454249 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.454250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.454262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.454343 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:02.454489 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.455038 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.456085 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.456439 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.456493 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.456500 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.456668 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.456694 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 06:33:02.456784 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.454032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39647 (local address 127.13.15.131:49325)
0929 06:33:02.454099 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.454113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.454190 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.454346 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.454348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.454350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:02.454430 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.454432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.455105 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.455105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.456006 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.456007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.456520 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.456520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.456594 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.456599 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.456655 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.456691 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 06:33:02.457244 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2390d67959403689f7f5409536f3cc" candidate_uuid: "0947463b223941e2bb5e1528ffba2886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4a01ec0adfc455091f33244f15b36a5" is_pre_election: true
I20260929 06:33:02.457414 30685 raft_consensus.cc:2454] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0947463b223941e2bb5e1528ffba2886 in term 0.
I20260929 06:33:02.457597 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.454195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44493 (local address 127.13.15.131:36641)
0929 06:33:02.454390 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.454401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.454443 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.455188 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.455188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.455190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:02.455252 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.455253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.455817 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.455818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.456675 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.456677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.457345 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.457346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.457412 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.457417 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.457479 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.457517 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260929 06:33:02.457643 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.454381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:36641 (local address 127.13.15.129:44493)
0929 06:33:02.454428 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.454434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.454442 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:02.455122 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.455123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.455131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.455169 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:02.455329 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.455744 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.456749 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.457265 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.457322 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.457328 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.457533 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.457580 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:33:02.457657 30753 leader_election.cc:302] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0947463b223941e2bb5e1528ffba2886, d4a01ec0adfc455091f33244f15b36a5; no voters:
I20260929 06:33:02.457813 30884 raft_consensus.cc:2781] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:02.457898 30884 raft_consensus.cc:482] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:02.457934 30884 raft_consensus.cc:3037] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:02.458025 30551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2390d67959403689f7f5409536f3cc" candidate_uuid: "0947463b223941e2bb5e1528ffba2886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca694749064b4ce38acb8ee0ac900d0e" is_pre_election: true
I20260929 06:33:02.458186 30551 raft_consensus.cc:2454] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0947463b223941e2bb5e1528ffba2886 in term 0.
I20260929 06:33:02.458735 30884 raft_consensus.cc:504] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.458861 30884 leader_election.cc:288] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [CANDIDATE]: Term 1 election: Requested vote from peers d4a01ec0adfc455091f33244f15b36a5 (127.13.15.130:39647), ca694749064b4ce38acb8ee0ac900d0e (127.13.15.129:44493)
I20260929 06:33:02.459030 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2390d67959403689f7f5409536f3cc" candidate_uuid: "0947463b223941e2bb5e1528ffba2886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4a01ec0adfc455091f33244f15b36a5"
I20260929 06:33:02.459043 30551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2390d67959403689f7f5409536f3cc" candidate_uuid: "0947463b223941e2bb5e1528ffba2886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca694749064b4ce38acb8ee0ac900d0e"
I20260929 06:33:02.459100 30685 raft_consensus.cc:3037] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:02.459139 30551 raft_consensus.cc:3037] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:02.459790 30685 raft_consensus.cc:2454] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0947463b223941e2bb5e1528ffba2886 in term 1.
I20260929 06:33:02.459899 30551 raft_consensus.cc:2454] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0947463b223941e2bb5e1528ffba2886 in term 1.
I20260929 06:33:02.459967 30753 leader_election.cc:302] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0947463b223941e2bb5e1528ffba2886, d4a01ec0adfc455091f33244f15b36a5; no voters:
I20260929 06:33:02.460071 30884 raft_consensus.cc:2781] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:02.460259 30884 raft_consensus.cc:686] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 1 LEADER]: Becoming Leader. State: Replica: 0947463b223941e2bb5e1528ffba2886, State: Running, Role: LEADER
I20260929 06:33:02.460354 30884 consensus_queue.cc:237] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } }
I20260929 06:33:02.461133 30406 catalog_manager.cc:5495] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0947463b223941e2bb5e1528ffba2886 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "0947463b223941e2bb5e1528ffba2886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0947463b223941e2bb5e1528ffba2886" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260929 06:33:02.468988 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.466998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.13.15.131:41829)
0929 06:33:02.467058 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.467060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.467067 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:33:02.467094 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.467095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.467108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.467166 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:02.467250 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.467666 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.468324 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.468489 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.468547 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.468666 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:02.468748 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:02.468804 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.468873 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.468888 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:33:02.470641 30551 raft_consensus.cc:1261] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 1 FOLLOWER]: Refusing update from remote peer 0947463b223941e2bb5e1528ffba2886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:02.470645 30685 raft_consensus.cc:1261] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 1 FOLLOWER]: Refusing update from remote peer 0947463b223941e2bb5e1528ffba2886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:02.470952 30889 consensus_queue.cc:1035] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a01ec0adfc455091f33244f15b36a5" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:02.471073 30884 consensus_queue.cc:1035] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca694749064b4ce38acb8ee0ac900d0e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:02.473271 30893 mvcc.cc:205] Tried to move back new op lower bound from 7334558033797586944 to 7334558033758629888. Current Snapshot: MvccSnapshot[applied={T|T < 7334558033797586944}]
I20260929 06:33:02.475502 30405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:02.475592 30405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:02.476285 30405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54782b71f30b43f9b5e4165810bb08cb: Sending DeleteTablet for 3 replicas of tablet bf2390d67959403689f7f5409536f3cc
I20260929 06:33:02.476581 30799 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2390d67959403689f7f5409536f3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:02 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:60994
I20260929 06:33:02.476584 30665 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2390d67959403689f7f5409536f3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:02 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:41500
I20260929 06:33:02.476794 30902 tablet_replica.cc:330] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: stopping tablet replica
I20260929 06:33:02.476795 30531 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2390d67959403689f7f5409536f3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:02 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:41668
I20260929 06:33:02.476845 30903 tablet_replica.cc:330] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: stopping tablet replica
I20260929 06:33:02.476974 30902 raft_consensus.cc:2229] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:02.477063 30904 tablet_replica.cc:330] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e: stopping tablet replica
I20260929 06:33:02.477144 30904 raft_consensus.cc:2229] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:02.477150 30902 raft_consensus.cc:2258] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:02.477237 30904 raft_consensus.cc:2258] T bf2390d67959403689f7f5409536f3cc P ca694749064b4ce38acb8ee0ac900d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:02.477278 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30468
I20260929 06:33:02.477581 30902 ts_tablet_manager.cc:1905] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:02.479738 30902 ts_tablet_manager.cc:1918] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:02.479837 30902 log.cc:1196] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/bf2390d67959403689f7f5409536f3cc
I20260929 06:33:02.480104 30902 ts_tablet_manager.cc:1939] T bf2390d67959403689f7f5409536f3cc P 0947463b223941e2bb5e1528ffba2886: Deleting consensus metadata
I20260929 06:33:02.480511 30392 catalog_manager.cc:4841] TS 0947463b223941e2bb5e1528ffba2886 (127.13.15.131:41829): tablet bf2390d67959403689f7f5409536f3cc (table test-table [id=e4124efd5f36463690c7e2753b668294]) successfully deleted
I20260929 06:33:02.482229 30903 raft_consensus.cc:2229] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:02.482360 30903 raft_consensus.cc:2258] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:02.482739 30903 ts_tablet_manager.cc:1905] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 06:33:02.483944 30391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:44493 (error 108)
I20260929 06:33:02.484127 30903 ts_tablet_manager.cc:1918] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:02.484200 30903 log.cc:1196] T bf2390d67959403689f7f5409536f3cc P d4a01ec0adfc455091f33244f15b36a5: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/bf2390d67959403689f7f5409536f3cc
I20260929 06:33:02.484419 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.484308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:44493 (local address 127.0.0.1:41676)
0929 06:33:02.484347 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.484387 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260929 06:33:02.484526 30391 catalog_manager.cc:4584] TS ca694749064b4ce38acb8ee0ac900d0e (127.13.15.129:44493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf2390d67959403689f7f5409536f3cc: Network error: Client connection negotiation failed: client connection to 127.13.15.129:44493: connect: Connection refused (error 111)
I20260929 06:33:02.484539 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30602
I20260929 06:33:02.489590 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30736
I20260929 06:33:02.489652 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.489557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39647 (local address 127.0.0.1:41516)
0929 06:33:02.489596 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.489622 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.130:39647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 06:33:02.494690 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30374
2026-09-29T06:33:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1861 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T06:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:02Z Disabled control of system clock
I20260929 06:33:02.521692 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:39001
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:39001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:02.599447 30909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.599694 30909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.599730 30909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.600948 30909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:02.600986 30909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.600998 30909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.601011 30909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:02.601022 30909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:02.602372 30909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:39001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:39001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.602605 30909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.602881 30909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:02.607839 30915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.608008 30909 server_base.cc:1034] running on GCE node
W20260929 06:33:02.607815 30914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.607808 30917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.608387 30909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.608724 30909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.609884 30909 hybrid_clock.cc:648] HybridClock initialized: now 1790663582609873 us; error 41 us; skew 500 ppm
I20260929 06:33:02.614571 30909 webserver.cc:460] Webserver started at http://127.13.15.190:33835/ using document root <none> and password file <none>
I20260929 06:33:02.614813 30909 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.614883 30909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.615018 30909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.616058 30909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "656cc2e378d345a0ba42691b233548f4"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "974ee6d2dedac6f23c849d20fe103c55"
server_key_iv: "508e21af674e73a91c26070062654e4e"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.616413 30909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "656cc2e378d345a0ba42691b233548f4"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "974ee6d2dedac6f23c849d20fe103c55"
server_key_iv: "508e21af674e73a91c26070062654e4e"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.617892 30909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 06:33:02.619294 30923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.619539 30909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:33:02.619629 30909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "656cc2e378d345a0ba42691b233548f4"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "974ee6d2dedac6f23c849d20fe103c55"
server_key_iv: "508e21af674e73a91c26070062654e4e"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.619719 30909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.644752 30909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.645097 30909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.645253 30909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.650295 30909 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:39001
I20260929 06:33:02.650337 30975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:39001 every 8 connection(s)
I20260929 06:33:02.650707 30909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:02.651993 30976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:02.655669 30976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: Bootstrap starting.
I20260929 06:33:02.655938 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 30909
I20260929 06:33:02.656059 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:02.656275 13374 external_mini_cluster.cc:1428] Setting key bd64ccf8f4f0ecd816aeb70ad43a167f
I20260929 06:33:02.656416 30976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:02.657049 30976 log.cc:824] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:02.657999 30976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: No bootstrap required, opened a new log
I20260929 06:33:02.659933 30976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } }
I20260929 06:33:02.660085 30976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:02.660146 30976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 656cc2e378d345a0ba42691b233548f4, State: Initialized, Role: FOLLOWER
I20260929 06:33:02.660264 30976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } }
I20260929 06:33:02.660343 30976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:02.660379 30976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:02.660426 30976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:02.661000 30976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } }
I20260929 06:33:02.661118 30976 leader_election.cc:302] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 656cc2e378d345a0ba42691b233548f4; no voters:
I20260929 06:33:02.661326 30976 leader_election.cc:288] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:02.661360 30981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:02.661562 30981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [term 1 LEADER]: Becoming Leader. State: Replica: 656cc2e378d345a0ba42691b233548f4, State: Running, Role: LEADER
I20260929 06:33:02.661608 30976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:02.661741 30981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } }
I20260929 06:33:02.662178 30981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 656cc2e378d345a0ba42691b233548f4. Latest consensus state: current_term: 1 leader_uuid: "656cc2e378d345a0ba42691b233548f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } } }
I20260929 06:33:02.662459 30981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:02.662380 30982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "656cc2e378d345a0ba42691b233548f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656cc2e378d345a0ba42691b233548f4" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39001 } } }
I20260929 06:33:02.662925 30982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:02.662992 30990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:02.663713 30990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:02.665530 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.656850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.13.15.190:39001)
0929 06:33:02.657044 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.657053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.657101 (+ 48us) server_negotiation.cc:407] Connection header received
0929 06:33:02.657898 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.657908 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.657994 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.658184 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.658450 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.660035 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.661323 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.661931 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.662062 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.664762 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.664766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.664767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:02.664996 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.665040 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:02.665055 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.665121 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.665216 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.665299 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260929 06:33:02.665654 30990 catalog_manager.cc:1347] Generated new cluster ID: 3a6e247808cb4b5f98ccc7eea1484fe4
I20260929 06:33:02.665693 30990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:02.678642 30990 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:02.679203 30990 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:02.685616 30990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: Generated new TSK 0
I20260929 06:33:02.685806 30990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:02.687422 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--builtin_ntp_servers=127.13.15.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:02.768153 31000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.768355 31000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.768378 31000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.769531 31000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.769574 31000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.769601 31000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:02.770927 31000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.771164 31000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.771477 31000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:02.777176 31005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.777202 31006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.777415 31000 server_base.cc:1034] running on GCE node
W20260929 06:33:02.777182 31008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.777773 31000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.778050 31000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.779234 31000 hybrid_clock.cc:648] HybridClock initialized: now 1790663582779214 us; error 36 us; skew 500 ppm
I20260929 06:33:02.780570 31000 webserver.cc:460] Webserver started at http://127.13.15.129:36871/ using document root <none> and password file <none>
I20260929 06:33:02.780835 31000 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.780908 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.781044 31000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.782099 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "d53026f37e3e45769a3809b76596dcd9"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "99993673c93a7cec0814adcb82357f92"
server_key_iv: "1e83c171843f7c8704f68455d876022a"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.782560 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "d53026f37e3e45769a3809b76596dcd9"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "99993673c93a7cec0814adcb82357f92"
server_key_iv: "1e83c171843f7c8704f68455d876022a"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.783978 31000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:02.784811 31014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.784976 31000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:02.785080 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "d53026f37e3e45769a3809b76596dcd9"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "99993673c93a7cec0814adcb82357f92"
server_key_iv: "1e83c171843f7c8704f68455d876022a"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.785174 31000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.808665 31000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.809012 31000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.809199 31000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.809463 31000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:02.809845 31000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:02.809885 31000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.809933 31000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:02.809964 31000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.816459 31000 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:36435
I20260929 06:33:02.816543 31127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:36435 every 8 connection(s)
I20260929 06:33:02.816895 31000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:02.822562 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.817651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56217 (local address 127.13.15.190:39001)
0929 06:33:02.817723 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.817725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.817990 (+ 265us) server_negotiation.cc:407] Connection header received
0929 06:33:02.818613 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.818614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.818628 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.818742 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.819430 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.819949 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.820961 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.821173 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.821265 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.822255 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.822256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.822257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:02.822262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.822278 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:02.822283 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.822343 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.822466 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.822495 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:33:02.822570 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31000
I20260929 06:33:02.822693 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:02.822803 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.817646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39001 (local address 127.13.15.129:56217)
0929 06:33:02.817860 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.817895 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.818481 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.818815 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.818818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.818826 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:02.819329 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.819333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.820075 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.820076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.820850 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.820852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.821277 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.821278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.821409 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.822007 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:02.822008 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:02.822036 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:02.822078 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:02.822349 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:02.822354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:02.822358 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.822440 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.822556 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260929 06:33:02.822947 13374 external_mini_cluster.cc:1428] Setting key b3b31c59e31056c6223e87e1a81f55b8
I20260929 06:33:02.823387 31128 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39001
I20260929 06:33:02.823516 31128 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:02.823730 31128 heartbeater.cc:511] Master 127.13.15.190:39001 requested a full tablet report, sending...
I20260929 06:33:02.824024 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--builtin_ntp_servers=127.13.15.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 06:33:02.824308 30940 ts_manager.cc:194] Registered new tserver with Master: d53026f37e3e45769a3809b76596dcd9 (127.13.15.129:36435)
I20260929 06:33:02.824939 30940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.129:56217
W20260929 06:33:02.903673 31131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:02.903880 31131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:02.903903 31131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:02.904996 31131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:02.905053 31131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:02.905078 31131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:02.906450 31131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:02.906634 31131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:02.906879 31131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:02.911752 31136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:02.911744 31139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.911896 31131 server_base.cc:1034] running on GCE node
W20260929 06:33:02.911744 31137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:02.912220 31131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:02.912473 31131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:02.913647 31131 hybrid_clock.cc:648] HybridClock initialized: now 1790663582913629 us; error 30 us; skew 500 ppm
I20260929 06:33:02.915035 31131 webserver.cc:460] Webserver started at http://127.13.15.130:36629/ using document root <none> and password file <none>
I20260929 06:33:02.915279 31131 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:02.915375 31131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:02.915524 31131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:02.916592 31131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "380fcfb5af9f46718f177c3e78b3cb72"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "5d3748872e5951192f9e54d89577cf1a"
server_key_iv: "85705f6bafa199ad8911ca8dad923ffd"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.916975 31131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "380fcfb5af9f46718f177c3e78b3cb72"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "5d3748872e5951192f9e54d89577cf1a"
server_key_iv: "85705f6bafa199ad8911ca8dad923ffd"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.918264 31131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:02.919001 31145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.919184 31131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:02.919287 31131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "380fcfb5af9f46718f177c3e78b3cb72"
format_stamp: "Formatted at 2026-09-29 06:33:02 on dist-test-slave-sm0p"
server_key: "5d3748872e5951192f9e54d89577cf1a"
server_key_iv: "85705f6bafa199ad8911ca8dad923ffd"
server_key_version: "encryptionkey@0"
I20260929 06:33:02.919376 31131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:02.935220 31131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:02.935549 31131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:02.935706 31131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:02.935938 31131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:02.936266 31131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:02.936319 31131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.936369 31131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:02.936400 31131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:02.942281 31131 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:39819
I20260929 06:33:02.942335 31258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:39819 every 8 connection(s)
I20260929 06:33:02.942719 31131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:33:02.947714 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.943530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:48887 (local address 127.13.15.190:39001)
0929 06:33:02.943642 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:02.943644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:02.943831 (+ 187us) server_negotiation.cc:407] Connection header received
0929 06:33:02.944297 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:02.944298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:02.944308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:02.944361 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:02.944978 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.945470 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.946477 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.946649 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.946697 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.947529 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:02.947530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:02.947530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:02.947534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:02.947543 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:02.947546 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:02.947581 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:02.947640 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 06:33:02.947657 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 06:33:02.947948 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:02.943494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39001 (local address 127.13.15.130:48887)
0929 06:33:02.943707 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:02.943741 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:02.944210 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:02.944414 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:02.944417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:02.944425 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:02.944910 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.944914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.945548 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.945549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.946410 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:02.946412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:02.946716 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:02.946717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:02.946818 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:02.947412 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:02.947412 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:02.947441 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:02.947462 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:02.947582 (+ 120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:02.947584 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:02.947587 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:02.947632 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 06:33:02.947704 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260929 06:33:02.948474 31259 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39001
I20260929 06:33:02.948585 31259 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:02.948760 31259 heartbeater.cc:511] Master 127.13.15.190:39001 requested a full tablet report, sending...
I20260929 06:33:02.949115 30940 ts_manager.cc:194] Registered new tserver with Master: 380fcfb5af9f46718f177c3e78b3cb72 (127.13.15.130:39819)
I20260929 06:33:02.949411 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31131
I20260929 06:33:02.949512 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:02.949620 30940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.130:48887
I20260929 06:33:02.949709 13374 external_mini_cluster.cc:1428] Setting key 771d62ad04737b3305b47ef2bf5de530
I20260929 06:33:02.950584 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--builtin_ntp_servers=127.13.15.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:03.028834 31262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:03.029026 31262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:03.029047 31262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:03.030371 31262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:03.030413 31262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:03.030438 31262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:03.031733 31262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:03.031971 31262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:03.032240 31262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:03.037068 31268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:03.037127 31262 server_base.cc:1034] running on GCE node
W20260929 06:33:03.037068 31267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:03.037050 31270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:03.037513 31262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:03.037741 31262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:03.038919 31262 hybrid_clock.cc:648] HybridClock initialized: now 1790663583038907 us; error 25 us; skew 500 ppm
I20260929 06:33:03.040088 31262 webserver.cc:460] Webserver started at http://127.13.15.131:43867/ using document root <none> and password file <none>
I20260929 06:33:03.040333 31262 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:03.040401 31262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:03.040531 31262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:03.041582 31262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "10fef445c3ab45748132e5482d98d014"
format_stamp: "Formatted at 2026-09-29 06:33:03 on dist-test-slave-sm0p"
server_key: "b4e928a24abb0908e21379fa095d7e7b"
server_key_iv: "ecd17b38bd5b058c89630878c0275ae9"
server_key_version: "encryptionkey@0"
I20260929 06:33:03.041957 31262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "10fef445c3ab45748132e5482d98d014"
format_stamp: "Formatted at 2026-09-29 06:33:03 on dist-test-slave-sm0p"
server_key: "b4e928a24abb0908e21379fa095d7e7b"
server_key_iv: "ecd17b38bd5b058c89630878c0275ae9"
server_key_version: "encryptionkey@0"
I20260929 06:33:03.043252 31262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:33:03.043977 31276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:03.044144 31262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:03.044238 31262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "10fef445c3ab45748132e5482d98d014"
format_stamp: "Formatted at 2026-09-29 06:33:03 on dist-test-slave-sm0p"
server_key: "b4e928a24abb0908e21379fa095d7e7b"
server_key_iv: "ecd17b38bd5b058c89630878c0275ae9"
server_key_version: "encryptionkey@0"
I20260929 06:33:03.044319 31262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:03.053725 31262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:03.053999 31262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:03.054184 31262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:03.054405 31262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:03.054698 31262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:03.054750 31262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:03.054798 31262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:03.054828 31262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:03.060664 31262 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:43931
I20260929 06:33:03.060699 31389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:43931 every 8 connection(s)
I20260929 06:33:03.061033 31262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:33:03.065099 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31262
I20260929 06:33:03.065205 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:03.065448 13374 external_mini_cluster.cc:1428] Setting key 9ec3028860912322c83953d023775451
I20260929 06:33:03.066504 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.061788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:33025 (local address 127.13.15.190:39001)
0929 06:33:03.061888 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.061891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.062106 (+ 215us) server_negotiation.cc:407] Connection header received
0929 06:33:03.062769 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.062770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.062779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.062845 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:03.063559 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.064040 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.064964 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.065275 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.065349 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.066248 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:03.066249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:03.066250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:03.066254 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:03.066269 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:03.066273 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:03.066325 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.066403 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.066428 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:33:03.066708 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.061762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39001 (local address 127.13.15.131:33025)
0929 06:33:03.061974 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.062019 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.062691 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.062893 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.062896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.062904 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:03.063483 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.063489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.064119 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.064120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.064887 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.064889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.065360 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.065361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.065470 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.066077 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:03.066077 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:03.066105 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:03.066127 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:03.066331 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:03.066336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:03.066339 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.066398 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.066495 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260929 06:33:03.067138 31390 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39001
I20260929 06:33:03.067236 31390 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:03.067407 31390 heartbeater.cc:511] Master 127.13.15.190:39001 requested a full tablet report, sending...
I20260929 06:33:03.067760 30940 ts_manager.cc:194] Registered new tserver with Master: 10fef445c3ab45748132e5482d98d014 (127.13.15.131:43931)
I20260929 06:33:03.068089 30940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.131:33025
I20260929 06:33:03.077256 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:03.081350 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.078680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.13.15.190:39001)
0929 06:33:03.078752 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.078754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.078884 (+ 130us) server_negotiation.cc:407] Connection header received
0929 06:33:03.078922 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.078922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.078932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.078982 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:03.079126 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.079589 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.080273 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.080444 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.080492 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.081153 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:03.081154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:03.081155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:03.081158 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:03.081168 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:03.081171 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:03.081211 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.081278 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.081293 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 06:33:03.082933 30940 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:03.084523 30940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:03.091028 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39819 (local address 127.0.0.1:43644)
0929 06:33:03.087602 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.087611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.087665 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.087839 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.087839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.087842 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:03.087879 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.087880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.089138 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.089139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.090018 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.090021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090733 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.090811 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.090825 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.090900 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.090954 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"thread_start_us":116,"threads_started":1}
I20260929 06:33:03.091040 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.13.15.130:39819)
0929 06:33:03.087104 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.087107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.087684 (+ 577us) server_negotiation.cc:407] Connection header received
0929 06:33:03.087729 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.087731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.087747 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.087826 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:03.087926 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.089055 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090099 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090629 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090730 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.090835 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.090918 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.090954 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260929 06:33:03.091203 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:43931 (local address 127.0.0.1:45028)
0929 06:33:03.086988 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.087030 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.087112 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.087313 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.087316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.087318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:03.087382 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.087384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.088567 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.088568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.089883 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.089885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090970 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.091021 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.091025 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.091053 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.091099 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":5,"thread_start_us":111,"threads_started":1}
I20260929 06:33:03.091562 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.13.15.131:43931)
0929 06:33:03.086979 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.086984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.087088 (+ 104us) server_negotiation.cc:407] Connection header received
0929 06:33:03.087196 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.087197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.087210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.087308 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:03.087449 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.088490 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090190 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090884 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.091321 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.091415 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.091438 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.091474 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260929 06:33:03.091737 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:36435 (local address 127.0.0.1:35942)
0929 06:33:03.087245 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.087255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.087301 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.087516 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.087517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.087519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:03.087554 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.087556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.088849 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.088850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.089883 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.089885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090938 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.091008 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.091012 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.091061 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.091234 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":138,"thread_start_us":60,"threads_started":1}
I20260929 06:33:03.091878 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.086850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.13.15.129:36435)
0929 06:33:03.087022 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.087027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.087365 (+ 338us) server_negotiation.cc:407] Connection header received
0929 06:33:03.087412 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.087415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.087432 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.087512 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:03.087638 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.088766 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.089974 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.090742 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.090812 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.090899 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.091387 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.091432 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260929 06:33:03.091835 31193 tablet_service.cc:1467] Processing CreateTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 (DEFAULT_TABLE table=test-table [id=c2efd7218cf548e58032f9a448d71fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:03.092240 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2deaae9d7374e3b9f413c16a0a23a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:03.092187 31324 tablet_service.cc:1467] Processing CreateTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 (DEFAULT_TABLE table=test-table [id=c2efd7218cf548e58032f9a448d71fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:03.092484 31324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2deaae9d7374e3b9f413c16a0a23a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:03.092425 31062 tablet_service.cc:1467] Processing CreateTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 (DEFAULT_TABLE table=test-table [id=c2efd7218cf548e58032f9a448d71fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:03.092660 31062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2deaae9d7374e3b9f413c16a0a23a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:03.095220 31404 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Bootstrap starting.
I20260929 06:33:03.095921 31404 tablet_bootstrap.cc:654] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:03.095988 31406 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Bootstrap starting.
I20260929 06:33:03.096243 31404 log.cc:824] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:03.096612 31406 tablet_bootstrap.cc:654] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:03.096906 31406 log.cc:824] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:03.097097 31404 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: No bootstrap required, opened a new log
I20260929 06:33:03.097177 31404 ts_tablet_manager.cc:1397] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:33:03.097236 31405 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Bootstrap starting.
I20260929 06:33:03.097688 31406 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: No bootstrap required, opened a new log
I20260929 06:33:03.097746 31405 tablet_bootstrap.cc:654] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:03.097848 31406 ts_tablet_manager.cc:1397] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:03.097961 31405 log.cc:824] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:03.098584 31405 tablet_bootstrap.cc:492] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: No bootstrap required, opened a new log
I20260929 06:33:03.098637 31405 ts_tablet_manager.cc:1397] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:03.099911 31404 raft_consensus.cc:348] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.100083 31404 raft_consensus.cc:374] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:03.100122 31404 raft_consensus.cc:729] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53026f37e3e45769a3809b76596dcd9, State: Initialized, Role: FOLLOWER
I20260929 06:33:03.100250 31404 consensus_queue.cc:260] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.100507 31404 ts_tablet_manager.cc:1428] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 06:33:03.100442 31405 raft_consensus.cc:348] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.100585 31405 raft_consensus.cc:374] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:03.100625 31405 raft_consensus.cc:729] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380fcfb5af9f46718f177c3e78b3cb72, State: Initialized, Role: FOLLOWER
I20260929 06:33:03.100631 31128 heartbeater.cc:503] Master 127.13.15.190:39001 was elected leader, sending a full tablet report...
I20260929 06:33:03.100741 31405 consensus_queue.cc:260] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.100993 31405 ts_tablet_manager.cc:1428] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:33:03.100994 31406 raft_consensus.cc:348] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.101140 31259 heartbeater.cc:503] Master 127.13.15.190:39001 was elected leader, sending a full tablet report...
I20260929 06:33:03.101146 31406 raft_consensus.cc:374] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:03.101178 31406 raft_consensus.cc:729] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fef445c3ab45748132e5482d98d014, State: Initialized, Role: FOLLOWER
I20260929 06:33:03.101276 31406 consensus_queue.cc:260] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.101523 31406 ts_tablet_manager.cc:1428] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 06:33:03.101598 31390 heartbeater.cc:503] Master 127.13.15.190:39001 was elected leader, sending a full tablet report...
W20260929 06:33:03.193284 31260 tablet.cc:2369] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:03.280115 31412 raft_consensus.cc:482] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:03.280244 31412 raft_consensus.cc:504] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.280664 31412 leader_election.cc:288] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53026f37e3e45769a3809b76596dcd9 (127.13.15.129:36435), 380fcfb5af9f46718f177c3e78b3cb72 (127.13.15.130:39819)
I20260929 06:33:03.284489 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.280955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:51873 (local address 127.13.15.130:39819)
0929 06:33:03.281027 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.281029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.281040 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:33:03.281075 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.281076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.281086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.281149 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:03.281323 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.282105 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.283584 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.284122 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.284198 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.284205 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.284383 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.284420 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:33:03.284489 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.280996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:36359 (local address 127.13.15.129:36435)
0929 06:33:03.281064 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.281066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.281275 (+ 209us) server_negotiation.cc:407] Connection header received
0929 06:33:03.281341 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.281342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.281358 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.281412 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:03.281515 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.282223 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.283583 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.284122 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.284198 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.284205 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.284383 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.284420 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:33:03.284489 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.280854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39819 (local address 127.13.15.131:51873)
0929 06:33:03.280920 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.280935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.280997 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.281171 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.281172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.281174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:03.281250 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.281252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.282223 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.282224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.283486 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.283489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.284210 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.284211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.284288 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.284292 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.284351 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.284400 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":15}
I20260929 06:33:03.284494 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.280858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:36435 (local address 127.13.15.131:36359)
0929 06:33:03.281198 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.281208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:03.281279 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:03.281411 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:03.281412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:03.281413 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:03.281455 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.281457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.282295 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.282296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.283487 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:03.283489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.284212 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.284213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:03.284299 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.284302 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:03.284351 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 06:33:03.284396 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":198,"thread_start_us":100,"threads_started":1}
I20260929 06:33:03.284864 31213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2deaae9d7374e3b9f413c16a0a23a99" candidate_uuid: "10fef445c3ab45748132e5482d98d014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380fcfb5af9f46718f177c3e78b3cb72" is_pre_election: true
I20260929 06:33:03.284864 31082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2deaae9d7374e3b9f413c16a0a23a99" candidate_uuid: "10fef445c3ab45748132e5482d98d014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53026f37e3e45769a3809b76596dcd9" is_pre_election: true
I20260929 06:33:03.285014 31213 raft_consensus.cc:2454] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fef445c3ab45748132e5482d98d014 in term 0.
I20260929 06:33:03.285014 31082 raft_consensus.cc:2454] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fef445c3ab45748132e5482d98d014 in term 0.
I20260929 06:33:03.285239 31277 leader_election.cc:302] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fef445c3ab45748132e5482d98d014, d53026f37e3e45769a3809b76596dcd9; no voters:
I20260929 06:33:03.285403 31412 raft_consensus.cc:2781] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:03.285476 31412 raft_consensus.cc:482] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:03.285509 31412 raft_consensus.cc:3037] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:03.286217 31412 raft_consensus.cc:504] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.286370 31412 leader_election.cc:288] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [CANDIDATE]: Term 1 election: Requested vote from peers d53026f37e3e45769a3809b76596dcd9 (127.13.15.129:36435), 380fcfb5af9f46718f177c3e78b3cb72 (127.13.15.130:39819)
I20260929 06:33:03.286528 31082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2deaae9d7374e3b9f413c16a0a23a99" candidate_uuid: "10fef445c3ab45748132e5482d98d014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53026f37e3e45769a3809b76596dcd9"
I20260929 06:33:03.286546 31213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2deaae9d7374e3b9f413c16a0a23a99" candidate_uuid: "10fef445c3ab45748132e5482d98d014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380fcfb5af9f46718f177c3e78b3cb72"
I20260929 06:33:03.286618 31082 raft_consensus.cc:3037] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:03.286675 31213 raft_consensus.cc:3037] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:03.287349 31082 raft_consensus.cc:2454] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fef445c3ab45748132e5482d98d014 in term 1.
I20260929 06:33:03.287407 31213 raft_consensus.cc:2454] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fef445c3ab45748132e5482d98d014 in term 1.
I20260929 06:33:03.287511 31277 leader_election.cc:302] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fef445c3ab45748132e5482d98d014, d53026f37e3e45769a3809b76596dcd9; no voters:
I20260929 06:33:03.287618 31412 raft_consensus.cc:2781] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:03.287757 31412 raft_consensus.cc:686] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 1 LEADER]: Becoming Leader. State: Replica: 10fef445c3ab45748132e5482d98d014, State: Running, Role: LEADER
I20260929 06:33:03.287860 31412 consensus_queue.cc:237] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } }
I20260929 06:33:03.288718 30940 catalog_manager.cc:5495] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10fef445c3ab45748132e5482d98d014 (127.13.15.131). New cstate: current_term: 1 leader_uuid: "10fef445c3ab45748132e5482d98d014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10fef445c3ab45748132e5482d98d014" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 43931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 } health_report { overall_health: UNKNOWN } } }
W20260929 06:33:03.311560 31391 tablet.cc:2369] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:03.315531 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.313563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.13.15.131:43931)
0929 06:33:03.313611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:03.313613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:03.313620 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:33:03.313648 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:03.313649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:03.313662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:03.313716 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:03.313814 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.314265 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.314920 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:03.315087 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:03.315132 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:03.315239 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:03.315323 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:03.315384 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:03.315412 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 06:33:03.315424 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:03.317075 31213 raft_consensus.cc:1261] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 1 FOLLOWER]: Refusing update from remote peer 10fef445c3ab45748132e5482d98d014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:03.317111 31082 raft_consensus.cc:1261] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 1 FOLLOWER]: Refusing update from remote peer 10fef445c3ab45748132e5482d98d014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:03.317443 31414 consensus_queue.cc:1035] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "380fcfb5af9f46718f177c3e78b3cb72" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 06:33:03.317533 31129 tablet.cc:2369] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:03.317584 31412 consensus_queue.cc:1035] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53026f37e3e45769a3809b76596dcd9" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 36435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:03.320724 31418 mvcc.cc:205] Tried to move back new op lower bound from 7334558037264297984 to 7334558037147701248. Current Snapshot: MvccSnapshot[applied={T|T < 7334558037264297984}]
I20260929 06:33:03.321987 30940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:03.322121 30940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:03.322922 30940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 656cc2e378d345a0ba42691b233548f4: Sending DeleteTablet for 3 replicas of tablet f2deaae9d7374e3b9f413c16a0a23a99
I20260929 06:33:03.323227 31062 tablet_service.cc:1514] Processing DeleteTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:03 UTC) from {username='slave'} at 127.0.0.1:35942
I20260929 06:33:03.323340 31193 tablet_service.cc:1514] Processing DeleteTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:03 UTC) from {username='slave'} at 127.0.0.1:43644
I20260929 06:33:03.323352 31324 tablet_service.cc:1514] Processing DeleteTablet for tablet f2deaae9d7374e3b9f413c16a0a23a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:03 UTC) from {username='slave'} at 127.0.0.1:45028
I20260929 06:33:03.323544 31427 tablet_replica.cc:330] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: stopping tablet replica
I20260929 06:33:03.323640 31427 raft_consensus.cc:2229] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:03.323660 31426 tablet_replica.cc:330] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9: stopping tablet replica
I20260929 06:33:03.323740 31427 raft_consensus.cc:2258] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:03.323731 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31000
I20260929 06:33:03.323875 31426 raft_consensus.cc:2229] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:03.324007 31426 raft_consensus.cc:2258] T f2deaae9d7374e3b9f413c16a0a23a99 P d53026f37e3e45769a3809b76596dcd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:03.324640 31427 ts_tablet_manager.cc:1905] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:03.325835 31428 tablet_replica.cc:330] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: stopping tablet replica
I20260929 06:33:03.325951 31428 raft_consensus.cc:2229] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:03.326043 31428 raft_consensus.cc:2258] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:03.326402 31427 ts_tablet_manager.cc:1918] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:03.326464 31427 log.cc:1196] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/f2deaae9d7374e3b9f413c16a0a23a99
I20260929 06:33:03.326471 31428 ts_tablet_manager.cc:1905] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:03.326725 31427 ts_tablet_manager.cc:1939] T f2deaae9d7374e3b9f413c16a0a23a99 P 10fef445c3ab45748132e5482d98d014: Deleting consensus metadata
I20260929 06:33:03.327080 30925 catalog_manager.cc:4841] TS 10fef445c3ab45748132e5482d98d014 (127.13.15.131:43931): tablet f2deaae9d7374e3b9f413c16a0a23a99 (table test-table [id=c2efd7218cf548e58032f9a448d71fe8]) successfully deleted
I20260929 06:33:03.327852 31428 ts_tablet_manager.cc:1918] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:03.327939 31428 log.cc:1196] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/f2deaae9d7374e3b9f413c16a0a23a99
I20260929 06:33:03.328150 31428 ts_tablet_manager.cc:1939] T f2deaae9d7374e3b9f413c16a0a23a99 P 380fcfb5af9f46718f177c3e78b3cb72: Deleting consensus metadata
I20260929 06:33:03.328531 30927 catalog_manager.cc:4841] TS 380fcfb5af9f46718f177c3e78b3cb72 (127.13.15.130:39819): tablet f2deaae9d7374e3b9f413c16a0a23a99 (table test-table [id=c2efd7218cf548e58032f9a448d71fe8]) successfully deleted
W20260929 06:33:03.331487 30924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:36435 (error 108)
I20260929 06:33:03.331947 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:03.331818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:36435 (local address 127.0.0.1:35958)
0929 06:33:03.331872 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:03.331910 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:36435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 06:33:03.331986 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31131
W20260929 06:33:03.333491 30924 catalog_manager.cc:4584] TS d53026f37e3e45769a3809b76596dcd9 (127.13.15.129:36435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2deaae9d7374e3b9f413c16a0a23a99: Network error: Client connection negotiation failed: client connection to 127.13.15.129:36435: connect: Connection refused (error 111)
I20260929 06:33:03.337725 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31262
I20260929 06:33:03.342573 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 30909
2026-09-29T06:33:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (846 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:03 dist-test-slave-sm0p krb5kdc[31432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:03 dist-test-slave-sm0p krb5kdc[31432](info): set up 2 sockets
Sep 29 06:33:03 dist-test-slave-sm0p krb5kdc[31432](info): commencing operation
krb5kdc: starting...
W20260929 06:33:04.362023 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:04.477490 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36755
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:36755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:04.555783 31445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:04.556006 31445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:04.556038 31445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:04.557039 31445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:04.557102 31445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:04.557124 31445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:04.557152 31445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:04.557180 31445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:04.558564 31445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:36755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:04.558838 31445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:04.559047 31445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:04.563915 31445 server_base.cc:1034] running on GCE node
W20260929 06:33:04.563851 31451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.563836 31450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.563844 31453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:04.564266 31445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:04.564489 31445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:04.565627 31445 hybrid_clock.cc:648] HybridClock initialized: now 1790663584565625 us; error 37 us; skew 500 ppm
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:04.567902 31445 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:04.568379 31445 webserver.cc:460] Webserver started at http://127.13.15.190:33971/ using document root <none> and password file <none>
I20260929 06:33:04.568608 31445 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:04.568675 31445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:04.568823 31445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:04.569794 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "117b2ea82ead46068bf0057dd6557a18"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "808ce8bef242a22e303fbaeb8dd883b2"
server_key_iv: "a13cd36e0573bcb0a17c8b8f17bb5b8e"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.570230 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "117b2ea82ead46068bf0057dd6557a18"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "808ce8bef242a22e303fbaeb8dd883b2"
server_key_iv: "a13cd36e0573bcb0a17c8b8f17bb5b8e"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.571444 31445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:04.572108 31460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.572324 31445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:04.572420 31445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "117b2ea82ead46068bf0057dd6557a18"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "808ce8bef242a22e303fbaeb8dd883b2"
server_key_iv: "a13cd36e0573bcb0a17c8b8f17bb5b8e"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.572505 31445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:04.584025 31445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:04.585636 31445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:04.585784 31445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:04.589613 31445 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:36755
I20260929 06:33:04.589643 31512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:36755 every 8 connection(s)
I20260929 06:33:04.589954 31445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:04.590736 31513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:04.591975 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31445
I20260929 06:33:04.592093 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:04.592343 13374 external_mini_cluster.cc:1428] Setting key aaa6c294d86888041a1590c1a7f2a998
I20260929 06:33:04.593338 31513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: Bootstrap starting.
I20260929 06:33:04.594051 31513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:04.594396 31513 log.cc:824] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:04.595089 31513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: No bootstrap required, opened a new log
I20260929 06:33:04.597314 31513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } }
I20260929 06:33:04.597505 31513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:04.597548 31513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117b2ea82ead46068bf0057dd6557a18, State: Initialized, Role: FOLLOWER
I20260929 06:33:04.597684 31513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } }
I20260929 06:33:04.597780 31513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:04.597837 31513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:04.597887 31513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:04.598605 31513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } }
I20260929 06:33:04.598688 31513 leader_election.cc:302] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 117b2ea82ead46068bf0057dd6557a18; no voters:
I20260929 06:33:04.598855 31513 leader_election.cc:288] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:04.598901 31518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:04.599040 31518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [term 1 LEADER]: Becoming Leader. State: Replica: 117b2ea82ead46068bf0057dd6557a18, State: Running, Role: LEADER
I20260929 06:33:04.599098 31513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:04.599120 31518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } }
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:04.599514 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "117b2ea82ead46068bf0057dd6557a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } } }
I20260929 06:33:04.599597 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:04.599813 31527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:04.600145 31520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 117b2ea82ead46068bf0057dd6557a18. Latest consensus state: current_term: 1 leader_uuid: "117b2ea82ead46068bf0057dd6557a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117b2ea82ead46068bf0057dd6557a18" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36755 } } }
I20260929 06:33:04.600286 31520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:04.600476 31527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:04.602746 31527 catalog_manager.cc:1347] Generated new cluster ID: 971b496ac9f642a4ae7d95aaaa155f86
I20260929 06:33:04.602794 31527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:04.603302 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:04.592864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.13.15.190:36755)
0929 06:33:04.593027 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:04.593034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:04.593057 (+ 23us) server_negotiation.cc:407] Connection header received
0929 06:33:04.593504 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:04.593517 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:04.593560 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:04.593686 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:04.594525 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.595443 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.596470 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.596849 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.596921 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:04.599822 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:04.599826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:04.599826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:04.599848 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:04.601592 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.602026 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.602029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.602029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.602191 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.602417 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.602419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.602419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.602712 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:04.602857 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:04.602995 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 06:33:04.603114 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260929 06:33:04.614300 31527 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:04.614885 31527 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:04.617620 31527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: Generated new TSK 0
I20260929 06:33:04.617781 31527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:33:04.645131 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:04.724086 31539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:04.724272 31539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:04.724294 31539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:04.725416 31539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:04.725455 31539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:04.725481 31539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:04.726938 31539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:04.727104 31539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:04.727336 31539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:04.732107 31545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.732141 31544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.732120 31547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:04.732343 31539 server_base.cc:1034] running on GCE node
I20260929 06:33:04.732578 31539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:04.732815 31539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:04.734002 31539 hybrid_clock.cc:648] HybridClock initialized: now 1790663584734000 us; error 43 us; skew 500 ppm
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:04.736522 31539 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:33:04.736886 31539 webserver.cc:460] Webserver started at http://127.13.15.129:36511/ using document root <none> and password file <none>
I20260929 06:33:04.737053 31539 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:04.737092 31539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:04.737174 31539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:04.738183 31539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "8d1e988f60744c9195e3a2e8128c4f36"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "916f346094e747844e25fb134194d878"
server_key_iv: "400163fbe6c4eb63a79b8216fde23ee9"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.738502 31539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "8d1e988f60744c9195e3a2e8128c4f36"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "916f346094e747844e25fb134194d878"
server_key_iv: "400163fbe6c4eb63a79b8216fde23ee9"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.739674 31539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:04.740312 31554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.740480 31539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:04.740561 31539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "8d1e988f60744c9195e3a2e8128c4f36"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "916f346094e747844e25fb134194d878"
server_key_iv: "400163fbe6c4eb63a79b8216fde23ee9"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.740614 31539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:04.751271 31539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:04.752969 31539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:04.753072 31539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:04.753240 31539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:04.753485 31539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:04.753515 31539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.753535 31539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:04.753547 31539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.759092 31539 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:34121
I20260929 06:33:04.759163 31667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:34121 every 8 connection(s)
I20260929 06:33:04.759474 31539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:04.759740 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31539
I20260929 06:33:04.759874 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:04.760107 13374 external_mini_cluster.cc:1428] Setting key bb451e4abecd6dae640fd1396bbef252
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
I20260929 06:33:04.769397 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:04.760404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:54767 (local address 127.13.15.190:36755)
0929 06:33:04.760465 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:04.760467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:04.760680 (+ 213us) server_negotiation.cc:407] Connection header received
0929 06:33:04.761220 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:04.761221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:04.761234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:04.761307 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:04.762377 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.762884 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.763913 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.764139 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.764224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:04.766562 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:04.766564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:04.766564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:04.766572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:04.767967 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.768413 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.768414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.768414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.768474 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.768747 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.768748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.768748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.768861 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:04.768933 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:04.769236 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 06:33:04.769323 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:33:04.769543 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:04.760380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:36755 (local address 127.13.15.129:54767)
0929 06:33:04.760548 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:04.760588 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:04.761121 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:04.761357 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:04.761361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:04.762001 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:04.762276 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:04.762281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.762991 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.762992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:04.763806 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:04.763808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.764237 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.764238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:04.764373 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:04.764919 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:04.764926 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:04.766466 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:04.768076 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:04.768084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:04.768086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:04.768334 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:04.768544 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:04.768545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:04.768545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:04.768672 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:04.768936 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:04.768943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:04.769076 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 06:33:04.769204 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 06:33:04.769357 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260929 06:33:04.770031 31668 heartbeater.cc:347] Connected to a master server at 127.13.15.190:36755
I20260929 06:33:04.770162 31668 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:04.770337 31668 heartbeater.cc:511] Master 127.13.15.190:36755 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
I20260929 06:33:04.770864 31477 ts_manager.cc:194] Registered new tserver with Master: 8d1e988f60744c9195e3a2e8128c4f36 (127.13.15.129:34121)
I20260929 06:33:04.771472 31477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:54767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:33:04.784636 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:04.863921 31673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:04.864104 31673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:04.864127 31673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:04.865267 31673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:04.865307 31673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:04.865332 31673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:04.866732 31673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:04.866945 31673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:04.867146 31673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:04.871800 31679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.871810 31681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:04.871834 31678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:04.871925 31673 server_base.cc:1034] running on GCE node
I20260929 06:33:04.872309 31673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:04.872493 31673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:04.873631 31673 hybrid_clock.cc:648] HybridClock initialized: now 1790663584873612 us; error 41 us; skew 500 ppm
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:04.876236 31673 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:33:04.876658 31673 webserver.cc:460] Webserver started at http://127.13.15.130:41969/ using document root <none> and password file <none>
I20260929 06:33:04.876861 31673 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:04.876899 31673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:04.877000 31673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:04.877897 31673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "286347387d274ccfa72303478d54795b"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "62c927ac65d65f588af9e01d4a0d3146"
server_key_iv: "9ad08bafd03a7616d7bb347080639c27"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.878235 31673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "286347387d274ccfa72303478d54795b"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "62c927ac65d65f588af9e01d4a0d3146"
server_key_iv: "9ad08bafd03a7616d7bb347080639c27"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.879395 31673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:33:04.880025 31688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.880220 31673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:04.880285 31673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "286347387d274ccfa72303478d54795b"
format_stamp: "Formatted at 2026-09-29 06:33:04 on dist-test-slave-sm0p"
server_key: "62c927ac65d65f588af9e01d4a0d3146"
server_key_iv: "9ad08bafd03a7616d7bb347080639c27"
server_key_version: "encryptionkey@0"
I20260929 06:33:04.880343 31673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:04.897928 31673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:04.900228 31673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:04.900384 31673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:04.900566 31673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:04.900868 31673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:04.900902 31673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.900954 31673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:04.900980 31673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:04.906591 31673 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:34223
I20260929 06:33:04.906665 31801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:34223 every 8 connection(s)
I20260929 06:33:04.906965 31673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:33:04.909217 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31673
I20260929 06:33:04.909313 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:04.909528 13374 external_mini_cluster.cc:1428] Setting key 48e30d864ffc7572a0d3ca3760271b6c
Sep 29 06:33:04 dist-test-slave-sm0p krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663584, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:04.916448 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:04.907721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:53855 (local address 127.13.15.190:36755)
0929 06:33:04.907801 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:04.907802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:04.907989 (+ 187us) server_negotiation.cc:407] Connection header received
0929 06:33:04.908437 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:04.908438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:04.908447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:04.908507 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:04.909480 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.910195 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.911152 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.911389 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.911478 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:04.913618 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:04.913619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:04.913619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:04.913627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:04.915075 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.915473 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.915474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.915474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.915535 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:04.915776 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:04.915777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:04.915777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:04.915897 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:04.915970 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:04.916241 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 06:33:04.916361 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 06:33:04.916602 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:04.907705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:36755 (local address 127.13.15.130:53855)
0929 06:33:04.907867 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:04.907907 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:04.908360 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:04.908554 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:04.908557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:04.909102 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:04.909325 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:04.909332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.910269 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.910270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:04.911058 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:04.911060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:04.911489 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:04.911491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:04.911628 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:04.912166 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:04.912174 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:04.913520 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:04.915169 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:04.915177 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:04.915179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:04.915397 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:04.915600 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:04.915601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:04.915601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:04.915709 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:04.915962 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:04.915968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:04.916106 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 06:33:04.916210 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 06:33:04.916375 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
I20260929 06:33:04.917060 31802 heartbeater.cc:347] Connected to a master server at 127.13.15.190:36755
I20260929 06:33:04.917171 31802 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:04.917378 31802 heartbeater.cc:511] Master 127.13.15.190:36755 requested a full tablet report, sending...
I20260929 06:33:04.917729 31477 ts_manager.cc:194] Registered new tserver with Master: 286347387d274ccfa72303478d54795b (127.13.15.130:34223)
I20260929 06:33:04.918244 31477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:53855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:33:04.936602 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:05.016942 31807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:05.017143 31807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:05.017165 31807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:05.018244 31807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:05.018282 31807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:05.018308 31807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:05.019619 31807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:05.019780 31807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:05.019934 31807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:05.024689 31815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.024792 31813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.024906 31807 server_base.cc:1034] running on GCE node
W20260929 06:33:05.024695 31812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.025172 31807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:05.025355 31807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:05.026484 31807 hybrid_clock.cc:648] HybridClock initialized: now 1790663585026467 us; error 30 us; skew 500 ppm
Sep 29 06:33:05 dist-test-slave-sm0p krb5kdc[31432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663585, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:05.028945 31807 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:33:05.029402 31807 webserver.cc:460] Webserver started at http://127.13.15.131:42799/ using document root <none> and password file <none>
I20260929 06:33:05.029620 31807 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:05.029664 31807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:05.029829 31807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:05.030890 31807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "10e68ed2cf054f7ba030921606d58422"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "e182d442fe98e66a2e23d9f05fa80300"
server_key_iv: "038285b267023d28fd21b67eb91ff142"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.031256 31807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "10e68ed2cf054f7ba030921606d58422"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "e182d442fe98e66a2e23d9f05fa80300"
server_key_iv: "038285b267023d28fd21b67eb91ff142"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.032538 31807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:33:05.033234 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.033448 31807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:05.033536 31807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "10e68ed2cf054f7ba030921606d58422"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "e182d442fe98e66a2e23d9f05fa80300"
server_key_iv: "038285b267023d28fd21b67eb91ff142"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.033609 31807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:05.047267 31807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:05.049515 31807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:05.049620 31807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:05.049840 31807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:05.050176 31807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:05.050232 31807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.050272 31807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:05.050302 31807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.056028 31807 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:41687
I20260929 06:33:05.056063 31935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:41687 every 8 connection(s)
I20260929 06:33:05.056377 31807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:33:05.061518 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31807
I20260929 06:33:05.061640 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:05.061867 13374 external_mini_cluster.cc:1428] Setting key cba8fe68d4b2cc400409f3da7582292a
Sep 29 06:33:05 dist-test-slave-sm0p krb5kdc[31432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663585, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:05.065433 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.057082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:57005 (local address 127.13.15.190:36755)
0929 06:33:05.057149 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.057150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.057316 (+ 166us) server_negotiation.cc:407] Connection header received
0929 06:33:05.057762 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.057763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.057774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.057825 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.058796 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.059294 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.060209 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.060392 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.060441 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.062767 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.062768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.062769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:05.062777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.064019 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:05.064436 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:05.064437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:05.064438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:05.064505 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:05.064760 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:05.064761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:05.064762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:05.064892 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.064966 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.065245 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.065345 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 06:33:05.065625 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.057040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:36755 (local address 127.13.15.131:57005)
0929 06:33:05.057194 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.057234 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.057685 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.057887 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.057890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.058497 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:05.058711 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.058716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.059373 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.059374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.060135 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.060137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.060460 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.060461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.060600 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.061061 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:05.061068 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:05.062672 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:05.064109 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:05.064116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:05.064119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:05.064346 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:05.064574 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:05.064575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:05.064576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:05.064690 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:05.064971 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:05.064979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:05.065114 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.065222 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.065376 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260929 06:33:05.066037 31936 heartbeater.cc:347] Connected to a master server at 127.13.15.190:36755
I20260929 06:33:05.066176 31936 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:05.066360 31936 heartbeater.cc:511] Master 127.13.15.190:36755 requested a full tablet report, sending...
I20260929 06:33:05.066751 31477 ts_manager.cc:194] Registered new tserver with Master: 10e68ed2cf054f7ba030921606d58422 (127.13.15.131:41687)
I20260929 06:33:05.067124 31477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:57005
I20260929 06:33:05.073594 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:05.080816 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.075030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46810 (local address 127.13.15.190:36755)
0929 06:33:05.075087 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.075089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.075199 (+ 110us) server_negotiation.cc:407] Connection header received
0929 06:33:05.075236 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.075237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.075244 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.075290 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.075869 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.076430 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.077174 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.077350 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.077400 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.078654 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.078655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.078656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:05.078661 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.079811 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:05.080197 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:05.080198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:05.080198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:05.080246 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:05.080425 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:05.080425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:05.080425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:05.080540 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.080620 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.080711 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.080756 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:33:05.082528 31477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:05.084178 31477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:05.090469 31948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:34121 (local address 127.0.0.1:44710)
0929 06:33:05.086908 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.086914 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.086949 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.087147 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.087148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.087150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.087201 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.087203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088467 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.088468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.089521 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.089523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090170 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.090171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.090245 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.090260 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.090343 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.090399 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":74,"threads_started":1}
I20260929 06:33:05.090494 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.13.15.129:34121)
0929 06:33:05.086781 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.086784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.086986 (+ 202us) server_negotiation.cc:407] Connection header received
0929 06:33:05.087048 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.087051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.087065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.087137 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.087267 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088362 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.089604 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090069 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.090172 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.090234 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.090377 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.090418 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260929 06:33:05.090656 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:41687 (local address 127.0.0.1:34124)
0929 06:33:05.086471 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.086773 (+ 302us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.086824 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.087057 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.087061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.087064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.087146 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.087149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088649 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.088650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.089529 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.089531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090440 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.090441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.090504 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.090509 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.090539 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.090578 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260929 06:33:05.090889 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34223 (local address 127.0.0.1:45682)
0929 06:33:05.086618 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.086654 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.086720 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.087295 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.087296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.087298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.087322 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.087323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088205 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.088206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.089501 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.089504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090647 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.090648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.090704 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.090707 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.090757 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.090798 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":36,"thread_start_us":59,"threads_started":1}
I20260929 06:33:05.091115 31946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.13.15.130:34223)
0929 06:33:05.087060 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.087065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.087075 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:05.087114 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.087116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.087132 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.087206 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.087397 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088104 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.089593 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090506 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.090566 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.090617 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.090981 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.091039 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":103,"threads_started":1}
I20260929 06:33:05.091199 31602 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb4998662814354a5f407ac5bc9af0d (DEFAULT_TABLE table=test-table [id=c81b16e0c4f245b9b14aaa47c72f9871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.091557 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.086496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.13.15.131:41687)
0929 06:33:05.086642 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.086647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.086824 (+ 177us) server_negotiation.cc:407] Connection header received
0929 06:33:05.086910 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.086912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.086926 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.087493 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.087562 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.088558 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.089641 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.090350 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.091307 (+ 957us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.091396 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.091423 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.091465 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260929 06:33:05.091573 31602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fb4998662814354a5f407ac5bc9af0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.091734 31736 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb4998662814354a5f407ac5bc9af0d (DEFAULT_TABLE table=test-table [id=c81b16e0c4f245b9b14aaa47c72f9871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.092052 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fb4998662814354a5f407ac5bc9af0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.092213 31870 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb4998662814354a5f407ac5bc9af0d (DEFAULT_TABLE table=test-table [id=c81b16e0c4f245b9b14aaa47c72f9871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.092505 31870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fb4998662814354a5f407ac5bc9af0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.095067 31951 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: Bootstrap starting.
I20260929 06:33:05.095234 31950 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Bootstrap starting.
I20260929 06:33:05.095577 31952 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Bootstrap starting.
I20260929 06:33:05.095886 31950 tablet_bootstrap.cc:654] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.095889 31951 tablet_bootstrap.cc:654] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.096190 31952 tablet_bootstrap.cc:654] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.096413 31950 log.cc:824] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.096469 31951 log.cc:824] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.096817 31952 log.cc:824] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.097280 31951 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: No bootstrap required, opened a new log
I20260929 06:33:05.097292 31950 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: No bootstrap required, opened a new log
I20260929 06:33:05.097357 31950 ts_tablet_manager.cc:1397] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:33:05.097357 31951 ts_tablet_manager.cc:1397] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:05.097584 31952 tablet_bootstrap.cc:492] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: No bootstrap required, opened a new log
I20260929 06:33:05.097646 31952 ts_tablet_manager.cc:1397] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:05.099035 31951 raft_consensus.cc:348] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099198 31951 raft_consensus.cc:374] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.099241 31951 raft_consensus.cc:729] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e988f60744c9195e3a2e8128c4f36, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.099356 31951 consensus_queue.cc:260] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099326 31950 raft_consensus.cc:348] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099509 31950 raft_consensus.cc:374] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.099557 31950 raft_consensus.cc:729] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e68ed2cf054f7ba030921606d58422, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.099524 31952 raft_consensus.cc:348] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099658 31952 raft_consensus.cc:374] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.099689 31952 raft_consensus.cc:729] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286347387d274ccfa72303478d54795b, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.099654 31950 consensus_queue.cc:260] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099753 31951 ts_tablet_manager.cc:1428] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:33:05.099776 31952 consensus_queue.cc:260] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.099912 31668 heartbeater.cc:503] Master 127.13.15.190:36755 was elected leader, sending a full tablet report...
I20260929 06:33:05.099918 31950 ts_tablet_manager.cc:1428] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:33:05.100031 31952 ts_tablet_manager.cc:1428] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 06:33:05.100090 31802 heartbeater.cc:503] Master 127.13.15.190:36755 was elected leader, sending a full tablet report...
I20260929 06:33:05.100170 31936 heartbeater.cc:503] Master 127.13.15.190:36755 was elected leader, sending a full tablet report...
W20260929 06:33:05.157583 31803 tablet.cc:2369] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:05.200757 31956 raft_consensus.cc:482] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:05.200928 31956 raft_consensus.cc:504] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.201272 31956 leader_election.cc:288] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 286347387d274ccfa72303478d54795b (127.13.15.130:34223), 10e68ed2cf054f7ba030921606d58422 (127.13.15.131:41687)
I20260929 06:33:05.204336 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.201487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:41687 (local address 127.13.15.129:56939)
0929 06:33:05.201546 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.201560 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.201600 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.201889 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.201890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.201892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.201965 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.201966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.202671 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.202672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.203455 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.203457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.204045 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.204046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.204128 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.204131 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.204184 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.204234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 06:33:05.204339 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.201667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:56939 (local address 127.13.15.131:41687)
0929 06:33:05.201746 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.201748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.201758 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:05.201787 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.201788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.201798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.201884 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.202034 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.202593 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.203555 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.203958 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.204017 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.204023 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.204214 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.204267 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 06:33:05.204375 31946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.201534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:35317 (local address 127.13.15.130:34223)
0929 06:33:05.201580 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.201582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.201823 (+ 241us) server_negotiation.cc:407] Connection header received
0929 06:33:05.201929 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.201930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.201941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.201970 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.202200 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.202692 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.203634 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.204017 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.204091 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.204098 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.204258 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.204297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:33:05.204357 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.201499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34223 (local address 127.13.15.129:35317)
0929 06:33:05.201723 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.201740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.201852 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.202039 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.202040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.202041 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.202108 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.202110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.202784 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.202785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.203549 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.203551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.204105 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.204106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.204157 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.204161 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.204237 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.204273 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":98,"threads_started":1}
I20260929 06:33:05.204736 31890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb4998662814354a5f407ac5bc9af0d" candidate_uuid: "8d1e988f60744c9195e3a2e8128c4f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10e68ed2cf054f7ba030921606d58422" is_pre_election: true
I20260929 06:33:05.204926 31890 raft_consensus.cc:2454] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e988f60744c9195e3a2e8128c4f36 in term 0.
I20260929 06:33:05.204958 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb4998662814354a5f407ac5bc9af0d" candidate_uuid: "8d1e988f60744c9195e3a2e8128c4f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "286347387d274ccfa72303478d54795b" is_pre_election: true
I20260929 06:33:05.205119 31756 raft_consensus.cc:2454] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e988f60744c9195e3a2e8128c4f36 in term 0.
I20260929 06:33:05.205202 31557 leader_election.cc:302] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e68ed2cf054f7ba030921606d58422, 8d1e988f60744c9195e3a2e8128c4f36; no voters:
I20260929 06:33:05.205348 31956 raft_consensus.cc:2781] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:05.205412 31956 raft_consensus.cc:482] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:05.205436 31956 raft_consensus.cc:3037] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:05.206117 31956 raft_consensus.cc:504] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.206323 31956 leader_election.cc:288] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [CANDIDATE]: Term 1 election: Requested vote from peers 286347387d274ccfa72303478d54795b (127.13.15.130:34223), 10e68ed2cf054f7ba030921606d58422 (127.13.15.131:41687)
I20260929 06:33:05.206480 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb4998662814354a5f407ac5bc9af0d" candidate_uuid: "8d1e988f60744c9195e3a2e8128c4f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "286347387d274ccfa72303478d54795b"
I20260929 06:33:05.206483 31890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb4998662814354a5f407ac5bc9af0d" candidate_uuid: "8d1e988f60744c9195e3a2e8128c4f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10e68ed2cf054f7ba030921606d58422"
I20260929 06:33:05.206552 31756 raft_consensus.cc:3037] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:05.206552 31890 raft_consensus.cc:3037] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:05.207422 31890 raft_consensus.cc:2454] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e988f60744c9195e3a2e8128c4f36 in term 1.
I20260929 06:33:05.207422 31756 raft_consensus.cc:2454] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e988f60744c9195e3a2e8128c4f36 in term 1.
I20260929 06:33:05.207597 31557 leader_election.cc:302] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e68ed2cf054f7ba030921606d58422, 8d1e988f60744c9195e3a2e8128c4f36; no voters:
I20260929 06:33:05.207705 31956 raft_consensus.cc:2781] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:05.207850 31956 raft_consensus.cc:686] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e988f60744c9195e3a2e8128c4f36, State: Running, Role: LEADER
I20260929 06:33:05.207955 31956 consensus_queue.cc:237] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } }
I20260929 06:33:05.208726 31475 catalog_manager.cc:5495] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e988f60744c9195e3a2e8128c4f36 (127.13.15.129). New cstate: current_term: 1 leader_uuid: "8d1e988f60744c9195e3a2e8128c4f36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e988f60744c9195e3a2e8128c4f36" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34121 } health_report { overall_health: HEALTHY } } }
I20260929 06:33:05.235375 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.232914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.13.15.129:34121)
0929 06:33:05.232979 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.232982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.232990 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:05.233037 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.233038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.233055 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.233133 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:05.233264 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.233743 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.234551 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.234748 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.234824 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.235003 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:05.235106 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:05.235214 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.235264 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.235276 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 06:33:05.237263 31890 raft_consensus.cc:1261] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 1 FOLLOWER]: Refusing update from remote peer 8d1e988f60744c9195e3a2e8128c4f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:05.237273 31756 raft_consensus.cc:1261] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 1 FOLLOWER]: Refusing update from remote peer 8d1e988f60744c9195e3a2e8128c4f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:05.237597 31956 consensus_queue.cc:1035] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10e68ed2cf054f7ba030921606d58422" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:05.237772 31960 consensus_queue.cc:1035] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "286347387d274ccfa72303478d54795b" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:05.240252 31963 mvcc.cc:205] Tried to move back new op lower bound from 7334558045128945664 to 7334558045012357120. Current Snapshot: MvccSnapshot[applied={T|T < 7334558045128945664}]
I20260929 06:33:05.242774 31475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:05.242939 31475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:05.243693 31475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 117b2ea82ead46068bf0057dd6557a18: Sending DeleteTablet for 3 replicas of tablet 1fb4998662814354a5f407ac5bc9af0d
I20260929 06:33:05.244011 31870 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fb4998662814354a5f407ac5bc9af0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:34124
I20260929 06:33:05.244014 31602 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fb4998662814354a5f407ac5bc9af0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:44710
I20260929 06:33:05.244066 31736 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fb4998662814354a5f407ac5bc9af0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:05 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:45682
I20260929 06:33:05.244251 31972 tablet_replica.cc:330] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36: stopping tablet replica
I20260929 06:33:05.244354 31972 raft_consensus.cc:2229] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:05.244442 31972 raft_consensus.cc:2258] T 1fb4998662814354a5f407ac5bc9af0d P 8d1e988f60744c9195e3a2e8128c4f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:05.244540 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31539
W20260929 06:33:05.250262 31461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:34121 (error 108)
I20260929 06:33:05.250283 31973 tablet_replica.cc:330] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: stopping tablet replica
I20260929 06:33:05.250305 31974 tablet_replica.cc:330] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b: stopping tablet replica
I20260929 06:33:05.250399 31974 raft_consensus.cc:2229] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:05.250401 31973 raft_consensus.cc:2229] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:05.250422 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31673
I20260929 06:33:05.250507 31973 raft_consensus.cc:2258] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:05.250509 31974 raft_consensus.cc:2258] T 1fb4998662814354a5f407ac5bc9af0d P 286347387d274ccfa72303478d54795b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:05.251432 31973 ts_tablet_manager.cc:1905] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:05.252141 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.252014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:34121 (local address 127.0.0.1:44740)
0929 06:33:05.252063 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.252101 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:34121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:33:05.252341 31461 catalog_manager.cc:4584] TS 8d1e988f60744c9195e3a2e8128c4f36 (127.13.15.129:34121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fb4998662814354a5f407ac5bc9af0d: Network error: Client connection negotiation failed: client connection to 127.13.15.129:34121: connect: Connection refused (error 111)
I20260929 06:33:05.253468 31973 ts_tablet_manager.cc:1918] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:05.253527 31973 log.cc:1196] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/1fb4998662814354a5f407ac5bc9af0d
I20260929 06:33:05.253794 31973 ts_tablet_manager.cc:1939] T 1fb4998662814354a5f407ac5bc9af0d P 10e68ed2cf054f7ba030921606d58422: Deleting consensus metadata
I20260929 06:33:05.254204 31463 catalog_manager.cc:4841] TS 10e68ed2cf054f7ba030921606d58422 (127.13.15.131:41687): tablet 1fb4998662814354a5f407ac5bc9af0d (table test-table [id=c81b16e0c4f245b9b14aaa47c72f9871]) successfully deleted
I20260929 06:33:05.255415 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.255288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34223 (local address 127.0.0.1:45684)
0929 06:33:05.255338 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.255384 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.130:34223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 06:33:05.255424 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31807
I20260929 06:33:05.260164 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31445
2026-09-29T06:33:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1918 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T06:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:05Z Disabled control of system clock
I20260929 06:33:05.286945 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:41407
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:41407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:05.363607 31979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:05.363803 31979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:05.363822 31979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:05.364883 31979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:05.364914 31979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:05.364928 31979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:05.364939 31979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:05.364951 31979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:05.366216 31979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:41407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:41407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:05.366376 31979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:05.366585 31979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:05.371075 31987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.371117 31985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.371107 31984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.371294 31979 server_base.cc:1034] running on GCE node
I20260929 06:33:05.371532 31979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:05.371797 31979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:05.372939 31979 hybrid_clock.cc:648] HybridClock initialized: now 1790663585372923 us; error 38 us; skew 500 ppm
I20260929 06:33:05.374053 31979 webserver.cc:460] Webserver started at http://127.13.15.190:34361/ using document root <none> and password file <none>
I20260929 06:33:05.374321 31979 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:05.374378 31979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:05.374539 31979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:05.375574 31979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "360bcf9b25c94df0bb15e6d794ced682"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "2e284174482883670131b5483d3ae481"
server_key_iv: "c41fcdc17911c20f0791de94ac2bc0ca"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.375913 31979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "360bcf9b25c94df0bb15e6d794ced682"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "2e284174482883670131b5483d3ae481"
server_key_iv: "c41fcdc17911c20f0791de94ac2bc0ca"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.377116 31979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:05.377766 31993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.377936 31979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:05.378031 31979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "360bcf9b25c94df0bb15e6d794ced682"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "2e284174482883670131b5483d3ae481"
server_key_iv: "c41fcdc17911c20f0791de94ac2bc0ca"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.378104 31979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:05.396713 31979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:05.397001 31979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:05.397145 31979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:05.400874 31979 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:41407
I20260929 06:33:05.400905 32045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:41407 every 8 connection(s)
I20260929 06:33:05.401222 31979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:05.402107 32046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.404359 32046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: Bootstrap starting.
I20260929 06:33:05.404940 32046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.405227 32046 log.cc:824] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.405864 32046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: No bootstrap required, opened a new log
I20260929 06:33:05.407425 32046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } }
I20260929 06:33:05.407553 32046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.407599 32046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360bcf9b25c94df0bb15e6d794ced682, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.407771 32046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } }
I20260929 06:33:05.407848 32046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:05.407897 32046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:05.407946 32046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:05.408432 32046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } }
I20260929 06:33:05.408538 32046 leader_election.cc:302] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 360bcf9b25c94df0bb15e6d794ced682; no voters:
I20260929 06:33:05.408716 32046 leader_election.cc:288] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:05.408778 32049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:05.408960 32049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [term 1 LEADER]: Becoming Leader. State: Replica: 360bcf9b25c94df0bb15e6d794ced682, State: Running, Role: LEADER
I20260929 06:33:05.408987 32046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:05.409108 32049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } }
I20260929 06:33:05.409467 32050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "360bcf9b25c94df0bb15e6d794ced682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } } }
I20260929 06:33:05.409548 32051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 360bcf9b25c94df0bb15e6d794ced682. Latest consensus state: current_term: 1 leader_uuid: "360bcf9b25c94df0bb15e6d794ced682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360bcf9b25c94df0bb15e6d794ced682" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 41407 } } }
I20260929 06:33:05.409559 32050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:05.409993 32051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:05.410060 32058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:05.410561 32058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:05.410868 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 31979
I20260929 06:33:05.410986 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:05.411180 13374 external_mini_cluster.cc:1428] Setting key 04026b5e6202a94d2b1b9f621710ceab
I20260929 06:33:05.412281 32058 catalog_manager.cc:1347] Generated new cluster ID: 01b64659a3ce4f50b5503a3b38432429
I20260929 06:33:05.412334 32058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:05.417032 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.411776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.13.15.190:41407)
0929 06:33:05.411953 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.411960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.411992 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:33:05.412096 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.412103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.412166 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.412365 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.412591 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.413969 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.414998 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.415612 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.415734 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.416332 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.416336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.416337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:05.416538 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.416576 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:05.416589 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.416673 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.416744 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.416818 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260929 06:33:05.429387 32058 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:05.429903 32058 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:05.436681 32058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: Generated new TSK 0
I20260929 06:33:05.436852 32058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:05.440909 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--builtin_ntp_servers=127.13.15.148:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:05.521263 32070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:05.521463 32070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:05.521485 32070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:05.522699 32070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:05.522738 32070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:05.522765 32070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:05.524104 32070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:05.524348 32070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:05.524658 32070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:05.530740 32076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.530748 32075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.530802 32078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.531002 32070 server_base.cc:1034] running on GCE node
I20260929 06:33:05.531301 32070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:05.531570 32070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:05.532721 32070 hybrid_clock.cc:648] HybridClock initialized: now 1790663585532710 us; error 27 us; skew 500 ppm
I20260929 06:33:05.534049 32070 webserver.cc:460] Webserver started at http://127.13.15.129:34627/ using document root <none> and password file <none>
I20260929 06:33:05.534332 32070 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:05.534377 32070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:05.534533 32070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:05.535662 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "f995b134d8aa0547b821f457a3c2e619"
server_key_iv: "d44772815cd6322966773b794b77bfbe"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.536010 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "f995b134d8aa0547b821f457a3c2e619"
server_key_iv: "d44772815cd6322966773b794b77bfbe"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.537304 32070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:05.538046 32084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.538256 32070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:05.538345 32070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "f995b134d8aa0547b821f457a3c2e619"
server_key_iv: "d44772815cd6322966773b794b77bfbe"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.538399 32070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:05.555284 32070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:05.555545 32070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:05.555711 32070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:05.555969 32070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:05.556331 32070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:05.556373 32070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.556421 32070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:05.556448 32070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.562561 32070 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:41317
I20260929 06:33:05.562634 32197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:41317 every 8 connection(s)
I20260929 06:33:05.562984 32070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:05.565881 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32070
I20260929 06:33:05.565999 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:05.566256 13374 external_mini_cluster.cc:1428] Setting key d3bf9b1ef2802f6d920bde7d89e8cc33
I20260929 06:33:05.567226 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--builtin_ntp_servers=127.13.15.148:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 06:33:05.568996 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.563789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:52187 (local address 127.13.15.190:41407)
0929 06:33:05.563858 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.563860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.564048 (+ 188us) server_negotiation.cc:407] Connection header received
0929 06:33:05.564586 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.564587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.564615 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.564671 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.565332 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.565948 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.567345 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.567673 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.567763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.568741 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.568742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.568742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:05.568748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.568762 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:05.568766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.568804 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.568920 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.568942 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:33:05.569286 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.563755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41407 (local address 127.13.15.129:52187)
0929 06:33:05.563928 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.563961 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.564478 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.564735 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.564745 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.564755 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:05.565230 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.565234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.566061 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.566062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.567230 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.567234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.567790 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.567791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.567937 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.568554 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:05.568555 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:05.568596 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:05.568628 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:05.568829 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:05.568835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:05.568837 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.568926 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.569041 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260929 06:33:05.569837 32198 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41407
I20260929 06:33:05.569926 32198 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:05.570106 32198 heartbeater.cc:511] Master 127.13.15.190:41407 requested a full tablet report, sending...
I20260929 06:33:05.570700 32010 ts_manager.cc:194] Registered new tserver with Master: e2bed3b5e04f4a2e85f1ed5ed38c59de (127.13.15.129:41317)
I20260929 06:33:05.571322 32010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.129:52187
W20260929 06:33:05.647011 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:05.647233 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:05.647266 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:05.648425 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:05.648486 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:05.648526 32201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:05.649868 32201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:05.650177 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:05.650481 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:05.655300 32209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.655284 32207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.655408 32201 server_base.cc:1034] running on GCE node
W20260929 06:33:05.655284 32206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.655714 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:05.655953 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:05.657126 32201 hybrid_clock.cc:648] HybridClock initialized: now 1790663585657111 us; error 27 us; skew 500 ppm
I20260929 06:33:05.658483 32201 webserver.cc:460] Webserver started at http://127.13.15.130:35839/ using document root <none> and password file <none>
I20260929 06:33:05.658744 32201 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:05.658794 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:05.658890 32201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:05.659909 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "c6562fba77f140929a4c3a36ec35c614"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "9810ee708258179ca71220060f5533ed"
server_key_iv: "526064138ddefea0a15d924e6ad78250"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.660226 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "c6562fba77f140929a4c3a36ec35c614"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "9810ee708258179ca71220060f5533ed"
server_key_iv: "526064138ddefea0a15d924e6ad78250"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.661478 32201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:05.662343 32215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.662493 32201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:05.662580 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "c6562fba77f140929a4c3a36ec35c614"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "9810ee708258179ca71220060f5533ed"
server_key_iv: "526064138ddefea0a15d924e6ad78250"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.662633 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:05.672340 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:05.672609 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:05.672713 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:05.672892 32201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:05.673243 32201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:05.673280 32201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.673342 32201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:05.673369 32201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.678953 32201 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:41671
I20260929 06:33:05.678997 32328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:41671 every 8 connection(s)
I20260929 06:33:05.679383 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:33:05.682675 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32201
I20260929 06:33:05.682803 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:05.683019 13374 external_mini_cluster.cc:1428] Setting key b23ac45aa8723db68d380a2c257f19c7
I20260929 06:33:05.684015 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--builtin_ntp_servers=127.13.15.148:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 06:33:05.693135 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.680117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:40249 (local address 127.13.15.190:41407)
0929 06:33:05.680188 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.680191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.680430 (+ 239us) server_negotiation.cc:407] Connection header received
0929 06:33:05.680996 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.680997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.681007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.681059 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.681722 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.682355 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.683806 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.683990 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.684044 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.692852 (+ 8808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.692854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.692854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:05.692859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.692874 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:05.692878 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.692918 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.693046 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.693076 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 06:33:05.693478 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.680109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41407 (local address 127.13.15.130:40249)
0929 06:33:05.680312 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.680345 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.680888 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.681113 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.681116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.681124 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:05.681640 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.681647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.682466 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.682467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.683720 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.683723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.684089 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.684090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.684263 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.692647 (+ 8384us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:05.692648 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:05.692692 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:05.692730 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:05.692946 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:05.692954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:05.692957 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.693051 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.693191 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260929 06:33:05.694039 32329 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41407
I20260929 06:33:05.694172 32329 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:05.694370 32329 heartbeater.cc:511] Master 127.13.15.190:41407 requested a full tablet report, sending...
I20260929 06:33:05.694742 32010 ts_manager.cc:194] Registered new tserver with Master: c6562fba77f140929a4c3a36ec35c614 (127.13.15.130:41671)
I20260929 06:33:05.695245 32010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.130:40249
W20260929 06:33:05.767522 32332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:05.767735 32332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:05.767757 32332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:05.768930 32332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:05.768976 32332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:05.769003 32332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:05.770380 32332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:05.770570 32332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:05.770836 32332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:05.776103 32338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:05.776103 32340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.776120 32332 server_base.cc:1034] running on GCE node
W20260929 06:33:05.776125 32337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:05.776566 32332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:05.776811 32332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:05.777976 32332 hybrid_clock.cc:648] HybridClock initialized: now 1790663585777965 us; error 28 us; skew 500 ppm
I20260929 06:33:05.779328 32332 webserver.cc:460] Webserver started at http://127.13.15.131:36387/ using document root <none> and password file <none>
I20260929 06:33:05.779584 32332 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:05.779644 32332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:05.779874 32332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:05.780957 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "6c5ec503d8044077b5d12be857178f7a"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "0ea3fdf60e859a47e8a23aebcd3cfaeb"
server_key_iv: "c83d4c61de5ff4a62ac5fe3ab768c857"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.781327 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "6c5ec503d8044077b5d12be857178f7a"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "0ea3fdf60e859a47e8a23aebcd3cfaeb"
server_key_iv: "c83d4c61de5ff4a62ac5fe3ab768c857"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.782678 32332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:05.783430 32346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.783697 32332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:33:05.783782 32332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "6c5ec503d8044077b5d12be857178f7a"
format_stamp: "Formatted at 2026-09-29 06:33:05 on dist-test-slave-sm0p"
server_key: "0ea3fdf60e859a47e8a23aebcd3cfaeb"
server_key_iv: "c83d4c61de5ff4a62ac5fe3ab768c857"
server_key_version: "encryptionkey@0"
I20260929 06:33:05.783833 32332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:05.795640 32332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:05.795885 32332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:05.795984 32332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:05.796165 32332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:05.796446 32332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:05.796476 32332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.796501 32332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:05.796530 32332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:05.802203 32332 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:35359
I20260929 06:33:05.802304 32459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:35359 every 8 connection(s)
I20260929 06:33:05.802699 32332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:33:05.808301 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.803440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:60447 (local address 127.13.15.190:41407)
0929 06:33:05.803487 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.803489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.803780 (+ 291us) server_negotiation.cc:407] Connection header received
0929 06:33:05.804304 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.804305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.804318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.804366 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.805051 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.805562 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.806660 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.806871 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.806949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.807937 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.807939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.807939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:05.807943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.807961 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:05.807966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.808026 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.808129 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.808192 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:05.808450 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.803453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:41407 (local address 127.13.15.131:60447)
0929 06:33:05.803640 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.803680 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.804195 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.804434 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.804438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.804449 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:05.804952 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.804956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.805662 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.805663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.806567 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.806570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.806955 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.806956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.807088 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.807744 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:05.807745 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:05.807798 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:05.807825 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:05.808043 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:05.808048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:05.808052 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.808123 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.808231 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260929 06:33:05.809110 32460 heartbeater.cc:347] Connected to a master server at 127.13.15.190:41407
I20260929 06:33:05.809125 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32332
I20260929 06:33:05.809208 32460 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:05.809221 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:05.809408 13374 external_mini_cluster.cc:1428] Setting key 2489d7dc24afb06dc28810c1e716d0c1
I20260929 06:33:05.809414 32460 heartbeater.cc:511] Master 127.13.15.190:41407 requested a full tablet report, sending...
I20260929 06:33:05.809938 32009 ts_manager.cc:194] Registered new tserver with Master: 6c5ec503d8044077b5d12be857178f7a (127.13.15.131:35359)
I20260929 06:33:05.810474 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:05.810467 32009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.131:60447
I20260929 06:33:05.814674 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.811795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.13.15.190:41407)
0929 06:33:05.811860 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.811862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.811977 (+ 115us) server_negotiation.cc:407] Connection header received
0929 06:33:05.812044 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.812045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.812055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.812100 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:05.812279 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.812719 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.813460 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.813683 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.813734 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.814385 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:05.814386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:05.814386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:05.814390 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:05.814402 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:05.814405 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:05.814478 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.814543 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.814552 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 06:33:05.816212 32009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:05.817672 32009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:05.824179 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.819872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.13.15.130:41671)
0929 06:33:05.820724 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.820728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.820737 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:05.820775 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.820777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.820793 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.820860 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.820989 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.821672 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823296 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.823781 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823885 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.823976 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.824056 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.824093 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":118,"threads_started":1}
I20260929 06:33:05.824170 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.819758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:41671 (local address 127.0.0.1:52114)
0929 06:33:05.820064 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.820103 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.820165 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.820869 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.820870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.820872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.820917 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.820918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.821757 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.821759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.823211 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.823213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.823889 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.823965 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.823977 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.824045 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.824096 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":29,"thread_start_us":50,"threads_started":1}
I20260929 06:33:05.824361 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.820045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.13.15.131:35359)
0929 06:33:05.820200 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.820205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.820582 (+ 377us) server_negotiation.cc:407] Connection header received
0929 06:33:05.820654 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.820656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.820671 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.820745 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.820918 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.821953 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823086 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.823796 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823869 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.823958 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.824209 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.824246 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260929 06:33:05.824416 32469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.819955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35359 (local address 127.0.0.1:48352)
0929 06:33:05.820502 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.820533 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.820587 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.820803 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.820804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.820806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.820844 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.820846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.822028 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.822029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.823003 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.823005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.823995 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.824053 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.824057 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.824105 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.824153 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":55,"threads_started":1}
I20260929 06:33:05.824522 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.819767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41317 (local address 127.0.0.1:37748)
0929 06:33:05.820064 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:05.820103 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:05.820165 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:05.820479 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:05.820483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:05.820485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:05.820548 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.820549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.821904 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.821905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.823101 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:05.823104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.824039 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.824040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:05.824096 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.824099 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:05.824144 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 06:33:05.824173 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":39,"thread_start_us":81,"threads_started":1}
I20260929 06:33:05.824874 32263 tablet_service.cc:1467] Processing CreateTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb (DEFAULT_TABLE table=test-table [id=eb8a160d9a134b5084c1e3ecdab80dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.825013 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:05.819858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.13.15.129:41317)
0929 06:33:05.820298 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:05.820311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:05.820319 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:05.820357 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:05.820358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:05.820370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:05.820431 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:05.820644 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.821822 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823177 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:05.823882 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:05.823940 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:05.824012 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:05.824208 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 06:33:05.824937 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":100,"threads_started":1}
I20260929 06:33:05.825042 32394 tablet_service.cc:1467] Processing CreateTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb (DEFAULT_TABLE table=test-table [id=eb8a160d9a134b5084c1e3ecdab80dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.825248 32263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b2ae3ab1824af5aa42d217c4a970bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.825328 32394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b2ae3ab1824af5aa42d217c4a970bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.828653 32475 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Bootstrap starting.
I20260929 06:33:05.829048 32132 tablet_service.cc:1467] Processing CreateTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb (DEFAULT_TABLE table=test-table [id=eb8a160d9a134b5084c1e3ecdab80dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:05.829399 32475 tablet_bootstrap.cc:654] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.829417 32132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b2ae3ab1824af5aa42d217c4a970bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:05.829698 32475 log.cc:824] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.830010 32474 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Bootstrap starting.
I20260929 06:33:05.830559 32474 tablet_bootstrap.cc:654] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.830833 32474 log.cc:824] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.831378 32475 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: No bootstrap required, opened a new log
I20260929 06:33:05.831461 32475 ts_tablet_manager.cc:1397] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 06:33:05.831561 32474 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: No bootstrap required, opened a new log
I20260929 06:33:05.831633 32474 ts_tablet_manager.cc:1397] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:33:05.832418 32478 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Bootstrap starting.
I20260929 06:33:05.833079 32478 tablet_bootstrap.cc:654] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:05.833419 32478 log.cc:824] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:05.833952 32475 raft_consensus.cc:348] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.834120 32475 raft_consensus.cc:374] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.834180 32475 raft_consensus.cc:729] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6562fba77f140929a4c3a36ec35c614, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.834192 32478 tablet_bootstrap.cc:492] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: No bootstrap required, opened a new log
I20260929 06:33:05.834244 32478 ts_tablet_manager.cc:1397] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:05.834323 32475 consensus_queue.cc:260] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.834302 32474 raft_consensus.cc:348] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.834472 32474 raft_consensus.cc:374] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.834506 32474 raft_consensus.cc:729] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5ec503d8044077b5d12be857178f7a, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.834591 32475 ts_tablet_manager.cc:1428] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 06:33:05.834615 32474 consensus_queue.cc:260] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.834722 32329 heartbeater.cc:503] Master 127.13.15.190:41407 was elected leader, sending a full tablet report...
I20260929 06:33:05.834893 32474 ts_tablet_manager.cc:1428] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:33:05.834928 32460 heartbeater.cc:503] Master 127.13.15.190:41407 was elected leader, sending a full tablet report...
I20260929 06:33:05.836447 32478 raft_consensus.cc:348] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.836565 32478 raft_consensus.cc:374] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:05.836585 32478 raft_consensus.cc:729] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bed3b5e04f4a2e85f1ed5ed38c59de, State: Initialized, Role: FOLLOWER
I20260929 06:33:05.836735 32478 consensus_queue.cc:260] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:05.837030 32478 ts_tablet_manager.cc:1428] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:33:05.837045 32198 heartbeater.cc:503] Master 127.13.15.190:41407 was elected leader, sending a full tablet report...
W20260929 06:33:05.929998 32330 tablet.cc:2369] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:33:06.057420 32461 tablet.cc:2369] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:33:06.063707 32199 tablet.cc:2369] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:06.065490 32482 raft_consensus.cc:482] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:06.065591 32482 raft_consensus.cc:504] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:06.066027 32482 leader_election.cc:288] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6562fba77f140929a4c3a36ec35c614 (127.13.15.130:41671), 6c5ec503d8044077b5d12be857178f7a (127.13.15.131:35359)
I20260929 06:33:06.069109 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.066289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:41671 (local address 127.13.15.129:53609)
0929 06:33:06.066341 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:06.066354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:06.066410 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:06.066705 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:06.066707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:06.066709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:06.066789 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:06.066792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.067459 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.067461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:06.068270 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:06.068272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.068840 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:06.068913 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:06.068916 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:06.068969 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 06:33:06.069004 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":35}
I20260929 06:33:06.069118 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.066506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:53609 (local address 127.13.15.130:41671)
0929 06:33:06.066574 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:06.066576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:06.066586 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:06.066622 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:06.066624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:06.066633 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:06.066691 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:06.066871 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.067379 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068368 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.068752 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068815 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:06.068845 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:06.068993 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 06:33:06.069044 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:33:06.069169 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.066506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:44855 (local address 127.13.15.131:35359)
0929 06:33:06.066575 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:06.066577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:06.066592 (+ 15us) server_negotiation.cc:407] Connection header received
0929 06:33:06.066700 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:06.066701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:06.066710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:06.066771 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:06.066966 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.067403 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068412 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.068767 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068859 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:06.068867 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:06.069054 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 06:33:06.069095 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:33:06.069428 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.066413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35359 (local address 127.13.15.129:44855)
0929 06:33:06.066515 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:06.066548 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:06.066611 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:06.066788 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:06.066789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:06.066790 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:06.066885 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:06.066887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.067506 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.067507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:06.068290 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:06.068292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.068889 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.068890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:06.068976 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:06.068980 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:06.069033 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 06:33:06.069069 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":65,"threads_started":1}
I20260929 06:33:06.069495 32283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b2ae3ab1824af5aa42d217c4a970bb" candidate_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6562fba77f140929a4c3a36ec35c614" is_pre_election: true
I20260929 06:33:06.069635 32283 raft_consensus.cc:2454] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2bed3b5e04f4a2e85f1ed5ed38c59de in term 0.
I20260929 06:33:06.069928 32088 leader_election.cc:302] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6562fba77f140929a4c3a36ec35c614, e2bed3b5e04f4a2e85f1ed5ed38c59de; no voters:
I20260929 06:33:06.069922 32414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b2ae3ab1824af5aa42d217c4a970bb" candidate_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5ec503d8044077b5d12be857178f7a" is_pre_election: true
I20260929 06:33:06.070060 32414 raft_consensus.cc:2454] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2bed3b5e04f4a2e85f1ed5ed38c59de in term 0.
I20260929 06:33:06.070154 32482 raft_consensus.cc:2781] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:06.070236 32482 raft_consensus.cc:482] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:06.070268 32482 raft_consensus.cc:3037] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:06.071027 32482 raft_consensus.cc:504] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:06.071163 32482 leader_election.cc:288] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [CANDIDATE]: Term 1 election: Requested vote from peers c6562fba77f140929a4c3a36ec35c614 (127.13.15.130:41671), 6c5ec503d8044077b5d12be857178f7a (127.13.15.131:35359)
I20260929 06:33:06.071346 32414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b2ae3ab1824af5aa42d217c4a970bb" candidate_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5ec503d8044077b5d12be857178f7a"
I20260929 06:33:06.071362 32283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b2ae3ab1824af5aa42d217c4a970bb" candidate_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6562fba77f140929a4c3a36ec35c614"
I20260929 06:33:06.071424 32414 raft_consensus.cc:3037] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:06.071461 32283 raft_consensus.cc:3037] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:06.072182 32283 raft_consensus.cc:2454] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2bed3b5e04f4a2e85f1ed5ed38c59de in term 1.
I20260929 06:33:06.072191 32414 raft_consensus.cc:2454] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2bed3b5e04f4a2e85f1ed5ed38c59de in term 1.
I20260929 06:33:06.072358 32088 leader_election.cc:302] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6562fba77f140929a4c3a36ec35c614, e2bed3b5e04f4a2e85f1ed5ed38c59de; no voters:
I20260929 06:33:06.072463 32482 raft_consensus.cc:2781] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:06.072624 32482 raft_consensus.cc:686] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 1 LEADER]: Becoming Leader. State: Replica: e2bed3b5e04f4a2e85f1ed5ed38c59de, State: Running, Role: LEADER
I20260929 06:33:06.072700 32482 consensus_queue.cc:237] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } }
I20260929 06:33:06.073498 32009 catalog_manager.cc:5495] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de reported cstate change: term changed from 0 to 1, leader changed from <none> to e2bed3b5e04f4a2e85f1ed5ed38c59de (127.13.15.129). New cstate: current_term: 1 leader_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2bed3b5e04f4a2e85f1ed5ed38c59de" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 41317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 } health_report { overall_health: UNKNOWN } } }
I20260929 06:33:06.107467 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.105011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.13.15.129:41317)
0929 06:33:06.105087 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:06.105089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:06.105097 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:06.105145 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:06.105146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:06.105166 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:06.105249 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:06.105399 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.105892 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.106687 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:06.106890 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:06.106970 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:06.107116 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:06.107209 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:06.107326 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:06.107358 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 06:33:06.107368 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 06:33:06.109316 32283 raft_consensus.cc:1261] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 1 FOLLOWER]: Refusing update from remote peer e2bed3b5e04f4a2e85f1ed5ed38c59de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:06.109333 32414 raft_consensus.cc:1261] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 1 FOLLOWER]: Refusing update from remote peer e2bed3b5e04f4a2e85f1ed5ed38c59de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:06.109720 32482 consensus_queue.cc:1035] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6562fba77f140929a4c3a36ec35c614" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:06.109867 32484 consensus_queue.cc:1035] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c5ec503d8044077b5d12be857178f7a" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:06.112433 32488 mvcc.cc:205] Tried to move back new op lower bound from 7334558048700919808 to 7334558048554541056. Current Snapshot: MvccSnapshot[applied={T|T < 7334558048700919808}]
I20260929 06:33:06.112857 32489 mvcc.cc:205] Tried to move back new op lower bound from 7334558048700919808 to 7334558048554541056. Current Snapshot: MvccSnapshot[applied={T|T < 7334558048700919808}]
I20260929 06:33:06.115202 32009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:06.115291 32009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:06.116050 32009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 360bcf9b25c94df0bb15e6d794ced682: Sending DeleteTablet for 3 replicas of tablet 74b2ae3ab1824af5aa42d217c4a970bb
I20260929 06:33:06.116375 32263 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:06 UTC) from {username='slave'} at 127.0.0.1:52114
I20260929 06:33:06.116372 32132 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:06 UTC) from {username='slave'} at 127.0.0.1:37748
I20260929 06:33:06.116441 32394 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b2ae3ab1824af5aa42d217c4a970bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:06 UTC) from {username='slave'} at 127.0.0.1:48352
I20260929 06:33:06.116605 32497 tablet_replica.cc:330] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de: stopping tablet replica
I20260929 06:33:06.116672 32499 tablet_replica.cc:330] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: stopping tablet replica
I20260929 06:33:06.116737 32498 tablet_replica.cc:330] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: stopping tablet replica
I20260929 06:33:06.116770 32499 raft_consensus.cc:2229] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:06.116869 32499 raft_consensus.cc:2258] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:06.116863 32498 raft_consensus.cc:2229] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:06.116910 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32070
I20260929 06:33:06.116994 32497 raft_consensus.cc:2229] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:06.117113 32498 raft_consensus.cc:2258] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:06.117234 32497 raft_consensus.cc:2258] T 74b2ae3ab1824af5aa42d217c4a970bb P e2bed3b5e04f4a2e85f1ed5ed38c59de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:06.117862 32498 ts_tablet_manager.cc:1905] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:06.118290 32499 ts_tablet_manager.cc:1905] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:06.119935 32498 ts_tablet_manager.cc:1918] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:06.120016 32498 log.cc:1196] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/74b2ae3ab1824af5aa42d217c4a970bb
I20260929 06:33:06.120038 32499 ts_tablet_manager.cc:1918] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:06.120105 32499 log.cc:1196] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/74b2ae3ab1824af5aa42d217c4a970bb
I20260929 06:33:06.120253 32498 ts_tablet_manager.cc:1939] T 74b2ae3ab1824af5aa42d217c4a970bb P c6562fba77f140929a4c3a36ec35c614: Deleting consensus metadata
I20260929 06:33:06.120365 32499 ts_tablet_manager.cc:1939] T 74b2ae3ab1824af5aa42d217c4a970bb P 6c5ec503d8044077b5d12be857178f7a: Deleting consensus metadata
I20260929 06:33:06.120638 31997 catalog_manager.cc:4841] TS c6562fba77f140929a4c3a36ec35c614 (127.13.15.130:41671): tablet 74b2ae3ab1824af5aa42d217c4a970bb (table test-table [id=eb8a160d9a134b5084c1e3ecdab80dc6]) successfully deleted
I20260929 06:33:06.120750 31997 catalog_manager.cc:4841] TS 6c5ec503d8044077b5d12be857178f7a (127.13.15.131:35359): tablet 74b2ae3ab1824af5aa42d217c4a970bb (table test-table [id=eb8a160d9a134b5084c1e3ecdab80dc6]) successfully deleted
W20260929 06:33:06.122807 31996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:41317 (error 108)
I20260929 06:33:06.122985 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32201
I20260929 06:33:06.123570 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:06.123107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:41317 (local address 127.0.0.1:37760)
0929 06:33:06.123500 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:06.123537 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375}
W20260929 06:33:06.123792 31996 catalog_manager.cc:4584] TS e2bed3b5e04f4a2e85f1ed5ed38c59de (127.13.15.129:41317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74b2ae3ab1824af5aa42d217c4a970bb: Network error: Client connection negotiation failed: client connection to 127.13.15.129:41317: connect: Connection refused (error 111)
I20260929 06:33:06.127904 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32332
I20260929 06:33:06.132361 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 31979
2026-09-29T06:33:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:06 dist-test-slave-sm0p krb5kdc[32503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:06 dist-test-slave-sm0p krb5kdc[32503](info): set up 2 sockets
Sep 29 06:33:06 dist-test-slave-sm0p krb5kdc[32503](info): commencing operation
krb5kdc: starting...
W20260929 06:33:07.158106 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:07.268235 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:39523
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:39523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:07.343642 32516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:07.343878 32516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:07.343925 32516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:07.345034 32516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:07.345103 32516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:07.345126 32516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:07.345155 32516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:07.345203 32516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:07.346661 32516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:39523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:07.346915 32516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:07.347157 32516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:07.351886 32521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.351912 32522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:07.351936 32516 server_base.cc:1034] running on GCE node
W20260929 06:33:07.351961 32524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:07.352288 32516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:07.352533 32516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:07.353703 32516 hybrid_clock.cc:648] HybridClock initialized: now 1790663587353691 us; error 28 us; skew 500 ppm
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:07.356155 32516 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:07.356602 32516 webserver.cc:460] Webserver started at http://127.13.15.190:44333/ using document root <none> and password file <none>
I20260929 06:33:07.356858 32516 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:07.356925 32516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:07.357079 32516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:07.358072 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "39849b52e9d145f7ab3c9c0498e33c92"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "fc78c1a5fb1c304ec8ef25a0ee99a5af"
server_key_iv: "a14d157fbe82d0378aa3e5d5adf3dad1"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.358476 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "39849b52e9d145f7ab3c9c0498e33c92"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "fc78c1a5fb1c304ec8ef25a0ee99a5af"
server_key_iv: "a14d157fbe82d0378aa3e5d5adf3dad1"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.359694 32516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:07.360352 32531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.360512 32516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:07.360606 32516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "39849b52e9d145f7ab3c9c0498e33c92"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "fc78c1a5fb1c304ec8ef25a0ee99a5af"
server_key_iv: "a14d157fbe82d0378aa3e5d5adf3dad1"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.360688 32516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:07.374718 32516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:07.376359 32516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:07.376518 32516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:07.380425 32516 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:39523
I20260929 06:33:07.380472 32583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:39523 every 8 connection(s)
I20260929 06:33:07.380811 32516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:07.381625 32584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:07.382460 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32516
I20260929 06:33:07.382592 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:07.382817 13374 external_mini_cluster.cc:1428] Setting key d652eb8fd1361a64e2c50f8ac4b38f85
I20260929 06:33:07.384883 32584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: Bootstrap starting.
I20260929 06:33:07.385439 32584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:07.385732 32584 log.cc:824] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:07.386490 32584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: No bootstrap required, opened a new log
I20260929 06:33:07.388996 32584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } }
I20260929 06:33:07.389170 32584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:07.389197 32584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39849b52e9d145f7ab3c9c0498e33c92, State: Initialized, Role: FOLLOWER
I20260929 06:33:07.389341 32584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } }
I20260929 06:33:07.389413 32584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:07.389456 32584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:07.389503 32584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:07.390080 32584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } }
I20260929 06:33:07.390218 32584 leader_election.cc:302] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39849b52e9d145f7ab3c9c0498e33c92; no voters:
I20260929 06:33:07.390399 32584 leader_election.cc:288] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:07.390434 32589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:07.390591 32589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [term 1 LEADER]: Becoming Leader. State: Replica: 39849b52e9d145f7ab3c9c0498e33c92, State: Running, Role: LEADER
I20260929 06:33:07.390658 32584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:07.390712 32589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } }
I20260929 06:33:07.391055 32589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39849b52e9d145f7ab3c9c0498e33c92. Latest consensus state: current_term: 1 leader_uuid: "39849b52e9d145f7ab3c9c0498e33c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } } }
I20260929 06:33:07.391063 32590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39849b52e9d145f7ab3c9c0498e33c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39849b52e9d145f7ab3c9c0498e33c92" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 39523 } } }
I20260929 06:33:07.391144 32589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:07.391165 32590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:07.391396 32595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:07.391893 32595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:07.393455 32595 catalog_manager.cc:1347] Generated new cluster ID: cc2709ef00814380b9f8f2d2c96cb716
I20260929 06:33:07.393512 32595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:07.394011 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.383314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.13.15.190:39523)
0929 06:33:07.383505 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.383514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.383545 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:33:07.383940 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.383946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.383989 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.384094 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:07.384883 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.385832 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.386899 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.387268 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.387353 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.390183 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:07.390187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:07.390187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:07.390199 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:07.392412 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.392764 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.392767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.392767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.392863 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.393108 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.393109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.393110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.393421 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:07.393575 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.393696 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.393820 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
I20260929 06:33:07.403012 32595 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:07.403568 32595 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:07.407969 32595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: Generated new TSK 0
I20260929 06:33:07.408131 32595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
Entry for principal kudu/127.13.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab.
I20260929 06:33:07.434419 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:07.515019 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:07.515244 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:07.515272 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:07.516525 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:07.516571 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:07.516599 32610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:07.518244 32610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.129.keytab
--principal=kudu/127.13.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:07.518496 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:07.518759 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:07.523763 32616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.523775 32618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.523777 32615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:07.523893 32610 server_base.cc:1034] running on GCE node
I20260929 06:33:07.524183 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:07.524447 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:07.525621 32610 hybrid_clock.cc:648] HybridClock initialized: now 1790663587525594 us; error 39 us; skew 500 ppm
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:07.528165 32610 init.cc:377] Logged in from keytab as kudu/127.13.15.129@KRBTEST.COM (short username kudu)
I20260929 06:33:07.528579 32610 webserver.cc:460] Webserver started at http://127.13.15.129:33189/ using document root <none> and password file <none>
I20260929 06:33:07.528822 32610 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:07.528895 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:07.529036 32610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:07.530048 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "16ea4e3fae894d32b7bc67bec34f9b5e"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "bfe6b65e8e3250c3c6e063e0824bb8cd"
server_key_iv: "af41ec16840615cda5796ef130962fb7"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.530464 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "16ea4e3fae894d32b7bc67bec34f9b5e"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "bfe6b65e8e3250c3c6e063e0824bb8cd"
server_key_iv: "af41ec16840615cda5796ef130962fb7"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.531692 32610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:07.532352 32625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.532537 32610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:07.532626 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "16ea4e3fae894d32b7bc67bec34f9b5e"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "bfe6b65e8e3250c3c6e063e0824bb8cd"
server_key_iv: "af41ec16840615cda5796ef130962fb7"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.532704 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:07.546231 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:07.547977 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:07.548138 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:07.548379 32610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:07.548678 32610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:07.548733 32610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.548767 32610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:07.548796 32610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.554646 32610 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:39377
I20260929 06:33:07.554716 32738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:39377 every 8 connection(s)
I20260929 06:33:07.555063 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:07.559118 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32610
I20260929 06:33:07.559231 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:07.559453 13374 external_mini_cluster.cc:1428] Setting key 95cc9c74a4187ae9ecca49caa86192e7
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.129@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:07.565325 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.555847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:48019 (local address 127.13.15.190:39523)
0929 06:33:07.555896 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.555898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.556134 (+ 236us) server_negotiation.cc:407] Connection header received
0929 06:33:07.556678 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.556679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.556692 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.556743 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:07.557719 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.558315 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.559589 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.559867 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.559928 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.562254 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:07.562255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:07.562256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:07.562262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:07.563756 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.564307 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.564308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.564308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.564375 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.564691 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.564692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.564692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.564787 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:07.564841 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.565178 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.565244 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:07.565730 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.555834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39523 (local address 127.13.15.129:48019)
0929 06:33:07.556011 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.556047 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.556552 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.556817 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.556821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.557406 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:07.557602 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.557607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.558413 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.558416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.559478 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.559482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.559956 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.559957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.560126 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.560698 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:07.560707 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:07.562145 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:07.563839 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.563847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.563849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.564229 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.564454 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.564455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.564456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.564619 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.564850 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:07.564864 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:07.565036 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.565164 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.565487 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260929 06:33:07.566316 32739 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39523
I20260929 06:33:07.566431 32739 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:07.566625 32739 heartbeater.cc:511] Master 127.13.15.190:39523 requested a full tablet report, sending...
I20260929 06:33:07.567139 32548 ts_manager.cc:194] Registered new tserver with Master: 16ea4e3fae894d32b7bc67bec34f9b5e (127.13.15.129:39377)
I20260929 06:33:07.567711 32548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.129@KRBTEST.COM'} at 127.13.15.129:48019
WARNING: no policy specified for kudu/127.13.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
Entry for principal kudu/127.13.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab.
I20260929 06:33:07.584343 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:07.662758 32744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:07.662945 32744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:07.662969 32744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:07.664085 32744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:07.664124 32744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:07.664150 32744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:07.665570 32744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.130.keytab
--principal=kudu/127.13.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:07.665845 32744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:07.666083 32744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:07.670989 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.671000 32752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.670992 32750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:07.671202 32744 server_base.cc:1034] running on GCE node
I20260929 06:33:07.671458 32744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:07.671701 32744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:07.672861 32744 hybrid_clock.cc:648] HybridClock initialized: now 1790663587672839 us; error 47 us; skew 500 ppm
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:07.675355 32744 init.cc:377] Logged in from keytab as kudu/127.13.15.130@KRBTEST.COM (short username kudu)
I20260929 06:33:07.675791 32744 webserver.cc:460] Webserver started at http://127.13.15.130:46697/ using document root <none> and password file <none>
I20260929 06:33:07.676025 32744 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:07.676090 32744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:07.676268 32744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:07.677290 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "f80341bdc6e84fc89b96e31bb1b44835"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "0f7d5b848609d371ded93c1ef7caf73c"
server_key_iv: "757af9355e16604a0a62a6e2e9b7dc60"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.677655 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "f80341bdc6e84fc89b96e31bb1b44835"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "0f7d5b848609d371ded93c1ef7caf73c"
server_key_iv: "757af9355e16604a0a62a6e2e9b7dc60"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.679041 32744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:07.679776 32759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.679965 32744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:07.680060 32744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "f80341bdc6e84fc89b96e31bb1b44835"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "0f7d5b848609d371ded93c1ef7caf73c"
server_key_iv: "757af9355e16604a0a62a6e2e9b7dc60"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.680151 32744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:07.703545 32744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:07.705467 32744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:07.705631 32744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:07.705861 32744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:07.706180 32744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:07.706236 32744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.706276 32744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:07.706305 32744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.712256 32744 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:39793
I20260929 06:33:07.712304 404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:39793 every 8 connection(s)
I20260929 06:33:07.712636 32744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.130@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:07.718889 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 32744
I20260929 06:33:07.719009 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:07.719252 13374 external_mini_cluster.cc:1428] Setting key 255771aeac23f95bf4f31634dde0dd16
I20260929 06:33:07.721923 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.713357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:49809 (local address 127.13.15.190:39523)
0929 06:33:07.713422 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.713425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.713645 (+ 220us) server_negotiation.cc:407] Connection header received
0929 06:33:07.714117 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.714118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.714128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.714188 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:07.715098 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.715606 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.716581 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.716789 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.716845 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.718907 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:07.718909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:07.718909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:07.718917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:07.720500 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.720953 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.720954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.720954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.721016 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.721265 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.721266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.721266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.721392 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:07.721469 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.721749 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.721826 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:33:07.722164 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.713339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39523 (local address 127.13.15.130:49809)
0929 06:33:07.713510 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.713546 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.714035 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.714266 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.714269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.714821 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:07.715016 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.715021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.715693 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.715693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.716484 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.716486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.716867 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.716868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.716962 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.717411 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:07.717419 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:07.718804 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:07.720588 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.720595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.720597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.720862 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.721082 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.721083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.721083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.721193 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.721461 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:07.721468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:07.721607 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.721726 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.721893 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260929 06:33:07.722659 405 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39523
I20260929 06:33:07.722750 405 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:07.722947 405 heartbeater.cc:511] Master 127.13.15.190:39523 requested a full tablet report, sending...
I20260929 06:33:07.723317 32548 ts_manager.cc:194] Registered new tserver with Master: f80341bdc6e84fc89b96e31bb1b44835 (127.13.15.130:39793)
I20260929 06:33:07.723721 32548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.130@KRBTEST.COM'} at 127.13.15.130:49809
WARNING: no policy specified for kudu/127.13.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
Entry for principal kudu/127.13.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab.
I20260929 06:33:07.744621 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:07.827136 410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:07.827360 410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:07.827401 410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:07.828533 410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:07.828600 410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:07.828644 410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:07.830063 410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.131.keytab
--principal=kudu/127.13.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:39523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:07.830376 410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:07.830603 410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:07.835474 410 server_base.cc:1034] running on GCE node
W20260929 06:33:07.835435 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.835435 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:07.835425 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:07.835922 410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:07.836164 410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:07.837320 410 hybrid_clock.cc:648] HybridClock initialized: now 1790663587837302 us; error 35 us; skew 500 ppm
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:07.839743 410 init.cc:377] Logged in from keytab as kudu/127.13.15.131@KRBTEST.COM (short username kudu)
I20260929 06:33:07.840161 410 webserver.cc:460] Webserver started at http://127.13.15.131:37693/ using document root <none> and password file <none>
I20260929 06:33:07.840385 410 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:07.840451 410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:07.840593 410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:07.841544 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "28825c8dbd1d463788f0b754dd0d2fd2"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "595907984c8fac8a7e7a7ecfb9079b14"
server_key_iv: "02b47c1080c5e84bba68d7c7959a2a7e"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.841892 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "28825c8dbd1d463788f0b754dd0d2fd2"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "595907984c8fac8a7e7a7ecfb9079b14"
server_key_iv: "02b47c1080c5e84bba68d7c7959a2a7e"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.843194 410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:07.843873 425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.844033 410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:07.844129 410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "28825c8dbd1d463788f0b754dd0d2fd2"
format_stamp: "Formatted at 2026-09-29 06:33:07 on dist-test-slave-sm0p"
server_key: "595907984c8fac8a7e7a7ecfb9079b14"
server_key_iv: "02b47c1080c5e84bba68d7c7959a2a7e"
server_key_version: "encryptionkey@0"
I20260929 06:33:07.844211 410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:07.856874 410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:07.858685 410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:07.858835 410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:07.859050 410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:07.859336 410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:07.859390 410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.859429 410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:07.859457 410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:07.865266 410 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:35017
I20260929 06:33:07.865334 538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:35017 every 8 connection(s)
I20260929 06:33:07.865615 410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:33:07.868875 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 410
I20260929 06:33:07.869004 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:07.869186 13374 external_mini_cluster.cc:1428] Setting key 73732db266a586a0545054e5932db13e
Sep 29 06:33:07 dist-test-slave-sm0p krb5kdc[32503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663587, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.131@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:07.874557 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.866261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:40327 (local address 127.13.15.190:39523)
0929 06:33:07.866320 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.866322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.866519 (+ 197us) server_negotiation.cc:407] Connection header received
0929 06:33:07.867008 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.867009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.867021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.867083 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:07.867990 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.868502 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.869747 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.870016 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.870103 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.872020 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:07.872021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:07.872021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:07.872029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:07.873167 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.873552 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.873553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.873553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.873612 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.873901 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.873902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.873902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.874013 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:07.874082 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.874393 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.874470 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:33:07.874725 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.866251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:39523 (local address 127.13.15.131:40327)
0929 06:33:07.866411 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.866442 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.866904 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.867148 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.867151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.867710 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:07.867892 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.867896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.868596 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.868597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.869654 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.869656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.870112 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.870113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.870266 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.870649 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:07.870656 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:07.871901 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:07.873250 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.873257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.873259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.873474 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.873708 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:07.873709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:07.873709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:07.873820 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:07.874084 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:07.874090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:07.874267 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.874368 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.874510 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260929 06:33:07.875147 539 heartbeater.cc:347] Connected to a master server at 127.13.15.190:39523
I20260929 06:33:07.875217 539 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:07.875344 539 heartbeater.cc:511] Master 127.13.15.190:39523 requested a full tablet report, sending...
I20260929 06:33:07.875640 32548 ts_manager.cc:194] Registered new tserver with Master: 28825c8dbd1d463788f0b754dd0d2fd2 (127.13.15.131:35017)
I20260929 06:33:07.875933 32548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.15.131@KRBTEST.COM'} at 127.13.15.131:40327
I20260929 06:33:07.880836 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:07.887636 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.882118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.13.15.190:39523)
0929 06:33:07.882185 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.882186 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.882291 (+ 105us) server_negotiation.cc:407] Connection header received
0929 06:33:07.882330 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.882331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.882339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.882388 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:07.883031 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.883489 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.884227 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.884399 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.884449 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.885441 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:07.885442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:07.885442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:07.885448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:07.886730 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.887031 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.887032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.887032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.887075 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:07.887235 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:07.887235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:07.887235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:07.887349 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:07.887419 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.887505 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.887558 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 06:33:07.889178 32548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:07.890841 32548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:07.897192 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.892959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:39793 (local address 127.0.0.1:48316)
0929 06:33:07.893315 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.893325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.893408 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.893674 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.893677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.893680 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:07.893742 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.893744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.894814 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.894815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.896027 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.896029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.896841 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.896842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.896927 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.896940 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.897063 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.897116 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260929 06:33:07.897204 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.893068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.13.15.130:39793)
0929 06:33:07.893265 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.893271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.893380 (+ 109us) server_negotiation.cc:407] Connection header received
0929 06:33:07.893499 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.893502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.893520 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.893599 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:07.893847 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.894699 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.896112 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.896752 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.896835 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.896895 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.897081 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.897123 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260929 06:33:07.897572 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.893142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:39377 (local address 127.0.0.1:55926)
0929 06:33:07.893477 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.893486 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.893522 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.894125 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.894126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.894143 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:07.894197 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.894198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.895378 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.895379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.896395 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.896398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.897348 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.897348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.897410 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.897413 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.897467 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.897505 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":51,"threads_started":1}
I20260929 06:33:07.897635 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.893268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.13.15.129:39377)
0929 06:33:07.893746 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.893750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.893759 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:07.893795 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.893797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.893813 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.894104 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:07.894265 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.895279 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.896497 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.897200 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.897265 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.897317 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.897491 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.897538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":86,"threads_started":1}
I20260929 06:33:07.898065 339 tablet_service.cc:1467] Processing CreateTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a (DEFAULT_TABLE table=test-table [id=5f9b9d53ee864a92a402254bb2a7617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:07.898290 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.892959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35017 (local address 127.0.0.1:42562)
0929 06:33:07.893260 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:07.893300 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:07.893385 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:07.893839 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:07.893840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:07.893842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:07.893887 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.893888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.895186 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.895187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.896210 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:07.896212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.897144 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.897145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:07.897207 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.897211 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 06:33:07.897257 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:33:07.898220 (+ 963us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260929 06:33:07.898445 552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:07.893257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.13.15.131:35017)
0929 06:33:07.893458 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:07.893462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:07.893471 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:07.893504 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:07.893506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:07.893519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:07.893618 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:07.893967 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.895076 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.896291 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:07.896969 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:07.897057 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:07.897108 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:07.897480 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 06:33:07.897523 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260929 06:33:07.898478 339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75250eed9dfe43fcaec2103eceb9da7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:07.898438 32673 tablet_service.cc:1467] Processing CreateTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a (DEFAULT_TABLE table=test-table [id=5f9b9d53ee864a92a402254bb2a7617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:07.898767 32673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75250eed9dfe43fcaec2103eceb9da7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:07.899051 473 tablet_service.cc:1467] Processing CreateTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a (DEFAULT_TABLE table=test-table [id=5f9b9d53ee864a92a402254bb2a7617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:07.899343 473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75250eed9dfe43fcaec2103eceb9da7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:07.901480 553 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Bootstrap starting.
I20260929 06:33:07.902119 553 tablet_bootstrap.cc:654] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:07.902388 554 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Bootstrap starting.
I20260929 06:33:07.902577 555 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Bootstrap starting.
I20260929 06:33:07.902761 553 log.cc:824] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:07.903025 554 tablet_bootstrap.cc:654] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:07.903201 555 tablet_bootstrap.cc:654] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:07.903679 554 log.cc:824] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:07.903831 555 log.cc:824] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:07.904450 553 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: No bootstrap required, opened a new log
I20260929 06:33:07.904532 553 ts_tablet_manager.cc:1397] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 06:33:07.904559 554 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: No bootstrap required, opened a new log
I20260929 06:33:07.904627 554 ts_tablet_manager.cc:1397] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:07.904641 555 tablet_bootstrap.cc:492] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: No bootstrap required, opened a new log
I20260929 06:33:07.904692 555 ts_tablet_manager.cc:1397] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:07.905911 553 raft_consensus.cc:348] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.906071 553 raft_consensus.cc:374] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:07.906116 553 raft_consensus.cc:729] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ea4e3fae894d32b7bc67bec34f9b5e, State: Initialized, Role: FOLLOWER
I20260929 06:33:07.906246 553 consensus_queue.cc:260] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.906562 553 ts_tablet_manager.cc:1428] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 06:33:07.906562 32739 heartbeater.cc:503] Master 127.13.15.190:39523 was elected leader, sending a full tablet report...
I20260929 06:33:07.906591 555 raft_consensus.cc:348] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.906591 554 raft_consensus.cc:348] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.906754 554 raft_consensus.cc:374] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:07.906751 555 raft_consensus.cc:374] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:07.906786 554 raft_consensus.cc:729] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80341bdc6e84fc89b96e31bb1b44835, State: Initialized, Role: FOLLOWER
I20260929 06:33:07.906786 555 raft_consensus.cc:729] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28825c8dbd1d463788f0b754dd0d2fd2, State: Initialized, Role: FOLLOWER
I20260929 06:33:07.906904 555 consensus_queue.cc:260] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.906924 554 consensus_queue.cc:260] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:07.907176 554 ts_tablet_manager.cc:1428] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:33:07.907176 555 ts_tablet_manager.cc:1428] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 06:33:07.907192 539 heartbeater.cc:503] Master 127.13.15.190:39523 was elected leader, sending a full tablet report...
I20260929 06:33:07.907297 405 heartbeater.cc:503] Master 127.13.15.190:39523 was elected leader, sending a full tablet report...
W20260929 06:33:07.963232 406 tablet.cc:2369] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 06:33:08.055814 32740 tablet.cc:2369] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:08.086529 561 raft_consensus.cc:482] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:08.086668 561 raft_consensus.cc:504] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:08.087083 561 leader_election.cc:288] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28825c8dbd1d463788f0b754dd0d2fd2 (127.13.15.131:35017), 16ea4e3fae894d32b7bc67bec34f9b5e (127.13.15.129:39377)
I20260929 06:33:08.089982 552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.087445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:41957 (local address 127.13.15.131:35017)
0929 06:33:08.087489 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.087492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.087501 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:08.087531 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.087532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.087541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.087599 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.087760 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.088247 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089240 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.089618 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089707 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.089716 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.089882 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.089920 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 06:33:08.089972 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.087310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:35017 (local address 127.13.15.130:41957)
0929 06:33:08.087392 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.087407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.087474 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.087609 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.087610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.087611 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.087687 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.087689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.088335 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.088336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.089140 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.089142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.089706 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.089791 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.089794 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.089858 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.089900 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":28}
I20260929 06:33:08.090037 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.087416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:38785 (local address 127.13.15.129:39377)
0929 06:33:08.087465 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.087467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.087591 (+ 124us) server_negotiation.cc:407] Connection header received
0929 06:33:08.087665 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.087666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.087676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.087753 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.087917 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.088370 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089322 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.089682 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089766 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.089774 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.089925 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.089958 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:33:08.090111 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.087310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:39377 (local address 127.13.15.130:38785)
0929 06:33:08.087510 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.087520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.087608 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.087756 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.087757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.087758 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.087846 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.087847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.088481 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.088482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.089230 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.089232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.089776 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.089777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.089852 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.089855 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.089920 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.089958 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":31,"thread_start_us":108,"threads_started":1}
I20260929 06:33:08.090466 493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75250eed9dfe43fcaec2103eceb9da7a" candidate_uuid: "f80341bdc6e84fc89b96e31bb1b44835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" is_pre_election: true
I20260929 06:33:08.090533 32693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75250eed9dfe43fcaec2103eceb9da7a" candidate_uuid: "f80341bdc6e84fc89b96e31bb1b44835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" is_pre_election: true
I20260929 06:33:08.090636 493 raft_consensus.cc:2454] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80341bdc6e84fc89b96e31bb1b44835 in term 0.
I20260929 06:33:08.090718 32693 raft_consensus.cc:2454] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80341bdc6e84fc89b96e31bb1b44835 in term 0.
I20260929 06:33:08.090907 32760 leader_election.cc:302] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28825c8dbd1d463788f0b754dd0d2fd2, f80341bdc6e84fc89b96e31bb1b44835; no voters:
I20260929 06:33:08.091039 561 raft_consensus.cc:2781] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:08.091120 561 raft_consensus.cc:482] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:08.091146 561 raft_consensus.cc:3037] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.091943 561 raft_consensus.cc:504] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:08.092078 561 leader_election.cc:288] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [CANDIDATE]: Term 1 election: Requested vote from peers 28825c8dbd1d463788f0b754dd0d2fd2 (127.13.15.131:35017), 16ea4e3fae894d32b7bc67bec34f9b5e (127.13.15.129:39377)
I20260929 06:33:08.092252 493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75250eed9dfe43fcaec2103eceb9da7a" candidate_uuid: "f80341bdc6e84fc89b96e31bb1b44835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28825c8dbd1d463788f0b754dd0d2fd2"
I20260929 06:33:08.092260 32693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75250eed9dfe43fcaec2103eceb9da7a" candidate_uuid: "f80341bdc6e84fc89b96e31bb1b44835" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e"
I20260929 06:33:08.092357 493 raft_consensus.cc:3037] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.092352 32693 raft_consensus.cc:3037] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.093063 32693 raft_consensus.cc:2454] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80341bdc6e84fc89b96e31bb1b44835 in term 1.
I20260929 06:33:08.093180 493 raft_consensus.cc:2454] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80341bdc6e84fc89b96e31bb1b44835 in term 1.
I20260929 06:33:08.093235 32761 leader_election.cc:302] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16ea4e3fae894d32b7bc67bec34f9b5e, f80341bdc6e84fc89b96e31bb1b44835; no voters:
I20260929 06:33:08.093353 561 raft_consensus.cc:2781] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:08.093501 561 raft_consensus.cc:686] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 1 LEADER]: Becoming Leader. State: Replica: f80341bdc6e84fc89b96e31bb1b44835, State: Running, Role: LEADER
I20260929 06:33:08.093606 561 consensus_queue.cc:237] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } }
I20260929 06:33:08.094411 32547 catalog_manager.cc:5495] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 reported cstate change: term changed from 0 to 1, leader changed from <none> to f80341bdc6e84fc89b96e31bb1b44835 (127.13.15.130). New cstate: current_term: 1 leader_uuid: "f80341bdc6e84fc89b96e31bb1b44835" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80341bdc6e84fc89b96e31bb1b44835" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 39793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 } health_report { overall_health: UNKNOWN } } }
W20260929 06:33:08.116258 540 tablet.cc:2369] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:08.122759 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.120279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.13.15.130:39793)
0929 06:33:08.120361 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.120363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.120373 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:08.120412 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.120413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.120435 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.120522 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:08.120665 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.121130 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.121880 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.122108 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.122221 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.122386 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:08.122517 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:08.122614 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.122633 (+ 19us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.122653 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 06:33:08.124557 493 raft_consensus.cc:1261] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 1 FOLLOWER]: Refusing update from remote peer f80341bdc6e84fc89b96e31bb1b44835: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:08.124567 32693 raft_consensus.cc:1261] T 75250eed9dfe43fcaec2103eceb9da7a P 16ea4e3fae894d32b7bc67bec34f9b5e [term 1 FOLLOWER]: Refusing update from remote peer f80341bdc6e84fc89b96e31bb1b44835: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:08.124888 563 consensus_queue.cc:1035] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28825c8dbd1d463788f0b754dd0d2fd2" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:08.125011 561 consensus_queue.cc:1035] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16ea4e3fae894d32b7bc67bec34f9b5e" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:08.128548 566 mvcc.cc:205] Tried to move back new op lower bound from 7334558056955514880 to 7334558056831832064. Current Snapshot: MvccSnapshot[applied={T|T < 7334558056955514880 or (T in {7334558056955514880})}]
I20260929 06:33:08.129781 32547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:08.129879 32547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:08.130659 32547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39849b52e9d145f7ab3c9c0498e33c92: Sending DeleteTablet for 3 replicas of tablet 75250eed9dfe43fcaec2103eceb9da7a
I20260929 06:33:08.130991 339 tablet_service.cc:1514] Processing DeleteTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:48316
I20260929 06:33:08.131032 473 tablet_service.cc:1514] Processing DeleteTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:42562
I20260929 06:33:08.131057 32673 tablet_service.cc:1514] Processing DeleteTablet for tablet 75250eed9dfe43fcaec2103eceb9da7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='kudu', principal='kudu/127.13.15.190@KRBTEST.COM'} at 127.0.0.1:55926
I20260929 06:33:08.131286 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32610
I20260929 06:33:08.131392 576 tablet_replica.cc:330] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: stopping tablet replica
I20260929 06:33:08.131506 576 raft_consensus.cc:2229] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:08.131639 576 raft_consensus.cc:2258] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:08.132130 576 ts_tablet_manager.cc:1905] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:08.133782 575 tablet_replica.cc:330] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: stopping tablet replica
I20260929 06:33:08.133900 575 raft_consensus.cc:2229] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:08.133917 576 ts_tablet_manager.cc:1918] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:08.134006 575 raft_consensus.cc:2258] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:08.134095 576 log.cc:1196] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/75250eed9dfe43fcaec2103eceb9da7a
I20260929 06:33:08.134441 576 ts_tablet_manager.cc:1939] T 75250eed9dfe43fcaec2103eceb9da7a P 28825c8dbd1d463788f0b754dd0d2fd2: Deleting consensus metadata
I20260929 06:33:08.134680 575 ts_tablet_manager.cc:1905] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:08.134853 32532 catalog_manager.cc:4841] TS 28825c8dbd1d463788f0b754dd0d2fd2 (127.13.15.131:35017): tablet 75250eed9dfe43fcaec2103eceb9da7a (table test-table [id=5f9b9d53ee864a92a402254bb2a7617e]) successfully deleted
I20260929 06:33:08.136286 575 ts_tablet_manager.cc:1918] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:08.136361 575 log.cc:1196] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/75250eed9dfe43fcaec2103eceb9da7a
I20260929 06:33:08.136619 575 ts_tablet_manager.cc:1939] T 75250eed9dfe43fcaec2103eceb9da7a P f80341bdc6e84fc89b96e31bb1b44835: Deleting consensus metadata
I20260929 06:33:08.137151 32533 catalog_manager.cc:4841] TS f80341bdc6e84fc89b96e31bb1b44835 (127.13.15.130:39793): tablet 75250eed9dfe43fcaec2103eceb9da7a (table test-table [id=5f9b9d53ee864a92a402254bb2a7617e]) successfully deleted
W20260929 06:33:08.141069 32533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:39377 (error 108)
I20260929 06:33:08.141304 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32744
I20260929 06:33:08.141546 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.141432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:39377 (local address 127.0.0.1:55936)
0929 06:33:08.141481 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.141512 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 06:33:08.141642 32533 catalog_manager.cc:4584] TS 16ea4e3fae894d32b7bc67bec34f9b5e (127.13.15.129:39377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75250eed9dfe43fcaec2103eceb9da7a: Network error: Client connection negotiation failed: client connection to 127.13.15.129:39377: connect: Connection refused (error 111)
I20260929 06:33:08.146944 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 410
I20260929 06:33:08.152127 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 32516
2026-09-29T06:33:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2020 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T06:33:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:08Z Disabled control of system clock
I20260929 06:33:08.179440 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34481
--webserver_interface=127.13.15.190
--webserver_port=0
--builtin_ntp_servers=127.13.15.148:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:34481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:08.258208 582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:08.258423 582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:08.258445 582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:08.259526 582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:08.259560 582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:08.259573 582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:08.259585 582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:08.259598 582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:08.260859 582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:34481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:34481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:08.261044 582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:08.261281 582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:08.266095 587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.266105 590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.266155 582 server_base.cc:1034] running on GCE node
W20260929 06:33:08.266105 588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.266527 582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:08.266762 582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:08.267923 582 hybrid_clock.cc:648] HybridClock initialized: now 1790663588267905 us; error 27 us; skew 500 ppm
I20260929 06:33:08.269093 582 webserver.cc:460] Webserver started at http://127.13.15.190:44623/ using document root <none> and password file <none>
I20260929 06:33:08.269356 582 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:08.269403 582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:08.269573 582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:08.270676 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "33050646fa9347eb8064284b73982cd6"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "828d39dfb96870a39dd9b0ab81c350c8"
server_key_iv: "5ec05f4ac33395debfb1bf95f425b738"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.271020 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "33050646fa9347eb8064284b73982cd6"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "828d39dfb96870a39dd9b0ab81c350c8"
server_key_iv: "5ec05f4ac33395debfb1bf95f425b738"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.272316 582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:33:08.273059 596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.273252 582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:08.273334 582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "33050646fa9347eb8064284b73982cd6"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "828d39dfb96870a39dd9b0ab81c350c8"
server_key_iv: "5ec05f4ac33395debfb1bf95f425b738"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.273393 582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:08.307785 582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:08.308076 582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:08.308182 582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:08.312014 582 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:34481
I20260929 06:33:08.312047 648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:34481 every 8 connection(s)
I20260929 06:33:08.312423 582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:08.313444 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:08.313742 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 582
I20260929 06:33:08.313856 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:08.314080 13374 external_mini_cluster.cc:1428] Setting key a8a713f593425a89b7f39a81abe97ae2
I20260929 06:33:08.316087 649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: Bootstrap starting.
I20260929 06:33:08.316659 649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:08.316990 649 log.cc:824] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:08.317736 649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: No bootstrap required, opened a new log
I20260929 06:33:08.319660 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.314630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.13.15.190:34481)
0929 06:33:08.314850 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.314859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.314885 (+ 26us) server_negotiation.cc:407] Connection header received
0929 06:33:08.315290 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.315296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.315354 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.315513 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:08.315746 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.316657 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.317758 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.318241 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.318337 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.319003 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:08.319007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:08.319007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:08.319164 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:08.319195 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:08.319205 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:08.319252 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.319369 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.319426 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260929 06:33:08.320055 649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } }
I20260929 06:33:08.320204 649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:08.320227 649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33050646fa9347eb8064284b73982cd6, State: Initialized, Role: FOLLOWER
I20260929 06:33:08.320375 649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } }
I20260929 06:33:08.320456 649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:08.320497 649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:08.320544 649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.321076 649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } }
I20260929 06:33:08.321185 649 leader_election.cc:302] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33050646fa9347eb8064284b73982cd6; no voters:
I20260929 06:33:08.321380 649 leader_election.cc:288] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:08.321417 654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:08.321554 654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [term 1 LEADER]: Becoming Leader. State: Replica: 33050646fa9347eb8064284b73982cd6, State: Running, Role: LEADER
I20260929 06:33:08.321640 654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } }
I20260929 06:33:08.321656 649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:08.321988 655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33050646fa9347eb8064284b73982cd6. Latest consensus state: current_term: 1 leader_uuid: "33050646fa9347eb8064284b73982cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } } }
I20260929 06:33:08.322069 655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:08.322314 660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:08.322811 660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:08.323408 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33050646fa9347eb8064284b73982cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33050646fa9347eb8064284b73982cd6" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 34481 } } }
I20260929 06:33:08.323499 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:08.324245 660 catalog_manager.cc:1347] Generated new cluster ID: 8f1828ff127749c0b17d01781ffabe26
I20260929 06:33:08.324298 660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:08.338287 660 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:08.339010 660 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:08.346297 660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: Generated new TSK 0
I20260929 06:33:08.346496 660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:08.352291 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.129:0
--local_ip_for_outbound_sockets=127.13.15.129
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--builtin_ntp_servers=127.13.15.148:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:08.436290 673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:08.436559 673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:08.436587 673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:08.437839 673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:08.437912 673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:08.437939 673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.129
I20260929 06:33:08.439347 673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.15.129
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.129
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:08.439589 673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:08.439896 673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:08.445902 679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.446005 681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.446161 673 server_base.cc:1034] running on GCE node
W20260929 06:33:08.446156 678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.446460 673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:08.446756 673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:08.447916 673 hybrid_clock.cc:648] HybridClock initialized: now 1790663588447896 us; error 37 us; skew 500 ppm
I20260929 06:33:08.449208 673 webserver.cc:460] Webserver started at http://127.13.15.129:36261/ using document root <none> and password file <none>
I20260929 06:33:08.449451 673 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:08.449532 673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:08.449667 673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:08.450789 673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data/instance:
uuid: "2cf25092ceba4f5e837cc4b30edd5171"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "026c3151b8033d594d7f7ebd1e745384"
server_key_iv: "4da75cc689ff14f17d9e6fcba9a2f743"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.451156 673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance:
uuid: "2cf25092ceba4f5e837cc4b30edd5171"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "026c3151b8033d594d7f7ebd1e745384"
server_key_iv: "4da75cc689ff14f17d9e6fcba9a2f743"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.452478 673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 06:33:08.453228 687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.453379 673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:08.453469 673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal
uuid: "2cf25092ceba4f5e837cc4b30edd5171"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "026c3151b8033d594d7f7ebd1e745384"
server_key_iv: "4da75cc689ff14f17d9e6fcba9a2f743"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.453536 673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:08.469058 673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:08.469373 673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:08.469520 673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:08.469745 673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:08.470088 673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:08.470120 673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.470196 673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:08.470225 673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.476289 673 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.129:34239
I20260929 06:33:08.476341 800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.129:34239 every 8 connection(s)
I20260929 06:33:08.476732 673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/data/info.pb
I20260929 06:33:08.477186 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 673
I20260929 06:33:08.477312 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-0/wal/instance
I20260929 06:33:08.477593 13374 external_mini_cluster.cc:1428] Setting key 28461b7b9229177367555497345e79ae
I20260929 06:33:08.478762 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.130:0
--local_ip_for_outbound_sockets=127.13.15.130
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--builtin_ntp_servers=127.13.15.148:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 06:33:08.483353 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.477647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.129:38327 (local address 127.13.15.190:34481)
0929 06:33:08.477744 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.477746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.477926 (+ 180us) server_negotiation.cc:407] Connection header received
0929 06:33:08.478571 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.478573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.478592 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.478723 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:08.479454 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.480251 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.481812 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.482010 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.482083 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.483070 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:08.483071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:08.483071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:08.483076 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:08.483090 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:08.483094 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:08.483150 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.483266 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.483292 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 06:33:08.483669 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.477626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34481 (local address 127.13.15.129:38327)
0929 06:33:08.477800 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.477848 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.478471 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.478771 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.478774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.478782 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:08.479359 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.479364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.480349 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.480350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.481694 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.481697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.482109 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.482110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.482302 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.482883 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:08.482884 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:08.482927 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:08.482959 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:08.483169 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:08.483177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:08.483180 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.483260 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.483392 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260929 06:33:08.484185 801 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34481
I20260929 06:33:08.484301 801 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:08.484483 801 heartbeater.cc:511] Master 127.13.15.190:34481 requested a full tablet report, sending...
I20260929 06:33:08.485054 613 ts_manager.cc:194] Registered new tserver with Master: 2cf25092ceba4f5e837cc4b30edd5171 (127.13.15.129:34239)
I20260929 06:33:08.485670 613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.129:38327
W20260929 06:33:08.560065 804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:08.560326 804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:08.560366 804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:08.561534 804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:08.561602 804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:08.561645 804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.130
I20260929 06:33:08.563046 804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.15.130
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.130
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:08.563328 804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:08.563652 804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:08.568710 809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.568739 810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.568748 812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.569023 804 server_base.cc:1034] running on GCE node
I20260929 06:33:08.569264 804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:08.569514 804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:08.570672 804 hybrid_clock.cc:648] HybridClock initialized: now 1790663588570664 us; error 42 us; skew 500 ppm
I20260929 06:33:08.571903 804 webserver.cc:460] Webserver started at http://127.13.15.130:43829/ using document root <none> and password file <none>
I20260929 06:33:08.572154 804 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:08.572225 804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:08.572388 804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:08.573452 804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data/instance:
uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "1ee8e777f26f162aaf221b7c651cd736"
server_key_iv: "01c64ed4e13557a108bb9c0401e87c49"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.573822 804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance:
uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "1ee8e777f26f162aaf221b7c651cd736"
server_key_iv: "01c64ed4e13557a108bb9c0401e87c49"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.575168 804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:08.575906 818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.576082 804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:08.576180 804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal
uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "1ee8e777f26f162aaf221b7c651cd736"
server_key_iv: "01c64ed4e13557a108bb9c0401e87c49"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.576259 804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:08.587028 804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:08.587328 804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:08.587481 804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:08.587695 804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:08.588012 804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:08.588071 804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.588112 804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:08.588140 804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.593849 804 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.130:34513
I20260929 06:33:08.593895 931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.130:34513 every 8 connection(s)
I20260929 06:33:08.594233 804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/data/info.pb
I20260929 06:33:08.599206 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.595015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.130:44829 (local address 127.13.15.190:34481)
0929 06:33:08.595074 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.595077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.595288 (+ 211us) server_negotiation.cc:407] Connection header received
0929 06:33:08.595758 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.595759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.595768 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.595816 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:08.596452 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.596933 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.597875 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.598051 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.598099 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.599001 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:08.599002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:08.599002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:08.599006 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:08.599017 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:08.599021 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:08.599060 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.599130 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.599148 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 06:33:08.599457 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.594985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34481 (local address 127.13.15.130:44829)
0929 06:33:08.595169 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.595201 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.595678 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.595879 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.595882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.595890 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:08.596386 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.596390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.597011 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.597012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.597805 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.597807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.598123 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.598123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.598297 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.598871 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:08.598872 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:08.598901 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:08.598924 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:08.599070 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:08.599072 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:08.599075 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.599122 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.599197 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260929 06:33:08.599934 932 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34481
I20260929 06:33:08.600040 932 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:08.600267 932 heartbeater.cc:511] Master 127.13.15.190:34481 requested a full tablet report, sending...
I20260929 06:33:08.600647 613 ts_manager.cc:194] Registered new tserver with Master: 4df1cfaa6f5c4de2aa1b8686c4967c78 (127.13.15.130:34513)
I20260929 06:33:08.601032 613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.130:44829
I20260929 06:33:08.604099 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 804
I20260929 06:33:08.604210 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal/instance
I20260929 06:33:08.604442 13374 external_mini_cluster.cc:1428] Setting key 34c2cd5dd8453c00850831564f36fd1c
I20260929 06:33:08.605340 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.15.131:0
--local_ip_for_outbound_sockets=127.13.15.131
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--builtin_ntp_servers=127.13.15.148:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 06:33:08.685505 935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:08.685711 935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:08.685732 935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:08.686903 935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:08.686944 935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:08.686996 935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.15.131
I20260929 06:33:08.688318 935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.15.131
--webserver_port=0
--tserver_master_addrs=127.13.15.190:34481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.15.131
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:08.688555 935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:08.688846 935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:08.693573 940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.693580 943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:08.693640 941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:08.693811 935 server_base.cc:1034] running on GCE node
I20260929 06:33:08.694025 935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:08.694278 935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:08.695416 935 hybrid_clock.cc:648] HybridClock initialized: now 1790663588695392 us; error 42 us; skew 500 ppm
I20260929 06:33:08.696600 935 webserver.cc:460] Webserver started at http://127.13.15.131:42983/ using document root <none> and password file <none>
I20260929 06:33:08.696832 935 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:08.696878 935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:08.697019 935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:08.698033 935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data/instance:
uuid: "fc1bb905203645329e9f7d38961e635d"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "97b357ea050d19b5291a9080d6aea71e"
server_key_iv: "3f319132815013264bb7f2f1d5c48e21"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.698457 935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance:
uuid: "fc1bb905203645329e9f7d38961e635d"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "97b357ea050d19b5291a9080d6aea71e"
server_key_iv: "3f319132815013264bb7f2f1d5c48e21"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.699765 935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:08.700486 949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.700665 935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:08.700750 935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal
uuid: "fc1bb905203645329e9f7d38961e635d"
format_stamp: "Formatted at 2026-09-29 06:33:08 on dist-test-slave-sm0p"
server_key: "97b357ea050d19b5291a9080d6aea71e"
server_key_iv: "3f319132815013264bb7f2f1d5c48e21"
server_key_version: "encryptionkey@0"
I20260929 06:33:08.700822 935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:08.715117 935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:08.715423 935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:08.715576 935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:08.715802 935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 06:33:08.716146 935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 06:33:08.716200 935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.716241 935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 06:33:08.716270 935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:08.722296 935 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.131:40485
I20260929 06:33:08.722349 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.131:40485 every 8 connection(s)
I20260929 06:33:08.722687 935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/data/info.pb
I20260929 06:33:08.727543 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.723452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:44185 (local address 127.13.15.190:34481)
0929 06:33:08.723535 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.723537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.723736 (+ 199us) server_negotiation.cc:407] Connection header received
0929 06:33:08.724198 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.724199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.724209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.724257 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:08.724866 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.725347 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.726269 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.726454 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.726504 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.727310 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:08.727312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:08.727312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:08.727316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:08.727339 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:08.727343 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:08.727402 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.727466 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.727485 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 06:33:08.727783 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.723412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:34481 (local address 127.13.15.131:44185)
0929 06:33:08.723617 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.723650 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.724120 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.724318 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.724320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.724328 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:08.724801 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.724806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.725425 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.725426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.726201 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.726203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.726524 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.726524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.726626 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.727184 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 06:33:08.727184 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:08.727212 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 06:33:08.727233 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:08.727405 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:08.727408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:08.727410 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.727456 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.727530 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260929 06:33:08.728261 1063 heartbeater.cc:347] Connected to a master server at 127.13.15.190:34481
I20260929 06:33:08.728361 1063 heartbeater.cc:464] Registering TS with master...
I20260929 06:33:08.728536 1063 heartbeater.cc:511] Master 127.13.15.190:34481 requested a full tablet report, sending...
I20260929 06:33:08.728909 613 ts_manager.cc:194] Registered new tserver with Master: fc1bb905203645329e9f7d38961e635d (127.13.15.131:40485)
I20260929 06:33:08.729228 613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.15.131:44185
I20260929 06:33:08.730216 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 935
I20260929 06:33:08.730332 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal/instance
I20260929 06:33:08.730533 13374 external_mini_cluster.cc:1428] Setting key bd997dc02f27339f0330baaafc848d34
I20260929 06:33:08.731408 13374 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 06:33:08.735311 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.732789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.13.15.190:34481)
0929 06:33:08.732850 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.732852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.732898 (+ 46us) server_negotiation.cc:407] Connection header received
0929 06:33:08.732962 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.732963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.732970 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.733012 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:08.733151 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.733596 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.734332 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.734524 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.734579 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.735136 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:08.735137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:08.735138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:08.735142 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:08.735154 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:08.735158 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:08.735200 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.735245 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.735262 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:33:08.736873 613 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 06:33:08.738458 613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 06:33:08.744653 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.13.15.131:40485)
0929 06:33:08.741058 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.741062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.741070 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:08.741101 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.741103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.741116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.741313 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.741373 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.742382 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.743656 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744264 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.744355 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.744444 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.744537 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.744571 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":68,"threads_started":1}
I20260929 06:33:08.744643 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.131:40485 (local address 127.0.0.1:60962)
0929 06:33:08.740905 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.740914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.740948 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.741202 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.741203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.741205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.741248 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.741249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.742450 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.742451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.743544 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.743547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744357 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.744358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.744434 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.744447 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.744516 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.744570 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":52,"threads_started":1}
I20260929 06:33:08.744791 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:34239 (local address 127.0.0.1:35124)
0929 06:33:08.740768 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.740778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.740817 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.741013 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.741016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.741019 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.741075 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.741076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.742097 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.742099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.743432 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.743435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744554 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.744554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.744605 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.744608 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.744637 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.744671 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":43,"thread_start_us":48,"threads_started":1}
I20260929 06:33:08.744937 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34513 (local address 127.0.0.1:38292)
0929 06:33:08.740669 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.740708 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.740770 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.741104 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.741105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.741107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.741141 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.741143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.742408 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.742408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.743580 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.743582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744523 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.744525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.744588 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.744591 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.744623 (+ 32us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.744671 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":35,"thread_start_us":95,"threads_started":1}
I20260929 06:33:08.745244 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.13.15.130:34513)
0929 06:33:08.740930 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.740934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.740944 (+ 10us) server_negotiation.cc:407] Connection header received
0929 06:33:08.740982 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.740985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.741001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.741103 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.741201 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.742320 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.743672 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744415 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.744919 (+ 504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.745100 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.745128 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.745168 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":87,"threads_started":1}
I20260929 06:33:08.745270 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.740603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.13.15.129:34239)
0929 06:33:08.740806 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.740811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.740822 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:33:08.740899 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.740902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.740918 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.740997 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.741152 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.741984 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.743780 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.744468 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.745046 (+ 578us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.745134 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.745156 (+ 22us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.745193 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260929 06:33:08.745695 997 tablet_service.cc:1467] Processing CreateTablet for tablet b14f1a64fe524d8b94192e201151cc31 (DEFAULT_TABLE table=test-table [id=03615cfab18e40949fb4473f0d3c21e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:08.745859 735 tablet_service.cc:1467] Processing CreateTablet for tablet b14f1a64fe524d8b94192e201151cc31 (DEFAULT_TABLE table=test-table [id=03615cfab18e40949fb4473f0d3c21e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:08.746085 997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14f1a64fe524d8b94192e201151cc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:08.746249 735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14f1a64fe524d8b94192e201151cc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:08.746536 866 tablet_service.cc:1467] Processing CreateTablet for tablet b14f1a64fe524d8b94192e201151cc31 (DEFAULT_TABLE table=test-table [id=03615cfab18e40949fb4473f0d3c21e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 06:33:08.746851 866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14f1a64fe524d8b94192e201151cc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:08.748733 1077 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Bootstrap starting.
I20260929 06:33:08.749364 1077 tablet_bootstrap.cc:654] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:08.749686 1077 log.cc:824] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:08.750041 1078 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Bootstrap starting.
I20260929 06:33:08.750177 1080 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Bootstrap starting.
I20260929 06:33:08.750622 1077 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: No bootstrap required, opened a new log
I20260929 06:33:08.750701 1077 ts_tablet_manager.cc:1397] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 06:33:08.750794 1080 tablet_bootstrap.cc:654] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:08.751021 1080 log.cc:824] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:08.751637 1080 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: No bootstrap required, opened a new log
I20260929 06:33:08.751693 1080 ts_tablet_manager.cc:1397] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 06:33:08.752660 1077 raft_consensus.cc:348] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.752815 1077 raft_consensus.cc:374] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:08.752866 1077 raft_consensus.cc:729] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1bb905203645329e9f7d38961e635d, State: Initialized, Role: FOLLOWER
I20260929 06:33:08.752986 1077 consensus_queue.cc:260] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.753265 1077 ts_tablet_manager.cc:1428] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 06:33:08.753311 1063 heartbeater.cc:503] Master 127.13.15.190:34481 was elected leader, sending a full tablet report...
I20260929 06:33:08.753831 1080 raft_consensus.cc:348] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.753966 1080 raft_consensus.cc:374] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:08.753988 1080 raft_consensus.cc:729] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df1cfaa6f5c4de2aa1b8686c4967c78, State: Initialized, Role: FOLLOWER
I20260929 06:33:08.753993 1078 tablet_bootstrap.cc:654] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:08.754099 1080 consensus_queue.cc:260] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.754369 1078 log.cc:824] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:08.754402 1080 ts_tablet_manager.cc:1428] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 06:33:08.754477 932 heartbeater.cc:503] Master 127.13.15.190:34481 was elected leader, sending a full tablet report...
I20260929 06:33:08.755268 1078 tablet_bootstrap.cc:492] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: No bootstrap required, opened a new log
I20260929 06:33:08.755345 1078 ts_tablet_manager.cc:1397] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260929 06:33:08.757230 1078 raft_consensus.cc:348] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.757380 1078 raft_consensus.cc:374] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:08.757426 1078 raft_consensus.cc:729] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf25092ceba4f5e837cc4b30edd5171, State: Initialized, Role: FOLLOWER
I20260929 06:33:08.757540 1078 consensus_queue.cc:260] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.757776 1078 ts_tablet_manager.cc:1428] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 06:33:08.757863 801 heartbeater.cc:503] Master 127.13.15.190:34481 was elected leader, sending a full tablet report...
W20260929 06:33:08.844774 933 tablet.cc:2369] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:08.958819 1082 raft_consensus.cc:482] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 06:33:08.958954 1082 raft_consensus.cc:504] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.959347 1082 leader_election.cc:288] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df1cfaa6f5c4de2aa1b8686c4967c78 (127.13.15.130:34513), 2cf25092ceba4f5e837cc4b30edd5171 (127.13.15.129:34239)
I20260929 06:33:08.962095 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.959661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:48325 (local address 127.13.15.129:34239)
0929 06:33:08.959709 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.959711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.959722 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:33:08.959754 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.959755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.959765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.959844 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.960003 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.960479 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.961395 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.961737 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.961830 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.961839 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.961991 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.962027 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 06:33:08.962095 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.959552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:34239 (local address 127.13.15.131:48325)
0929 06:33:08.959613 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.959627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.959694 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.959847 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.959848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.959849 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.959929 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.959930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.960564 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.960565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.961294 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.961296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.961830 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.961831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.961904 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.961907 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.961981 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.962023 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 06:33:08.962459 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.959674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.15.131:51273 (local address 127.13.15.130:34513)
0929 06:33:08.959720 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.959723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.959940 (+ 217us) server_negotiation.cc:407] Connection header received
0929 06:33:08.960018 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.960019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.960034 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.960096 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 06:33:08.960248 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.960706 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.961600 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.961951 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.962034 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.962041 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.962348 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.962398 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 06:33:08.962508 755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14f1a64fe524d8b94192e201151cc31" candidate_uuid: "fc1bb905203645329e9f7d38961e635d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf25092ceba4f5e837cc4b30edd5171" is_pre_election: true
I20260929 06:33:08.962704 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.959553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.130:34513 (local address 127.13.15.131:51273)
0929 06:33:08.959858 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.959867 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:08.959946 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:08.960097 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:08.960098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:08.960099 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 06:33:08.960175 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.960177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.960788 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.960790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.961508 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:08.961510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.962032 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.962033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:08.962110 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.962113 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 06:33:08.962581 (+ 468us) client_negotiation.cc:240] Negotiation successful
0929 06:33:08.962621 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":96,"thread_start_us":145,"threads_started":1}
I20260929 06:33:08.962723 755 raft_consensus.cc:2454] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1bb905203645329e9f7d38961e635d in term 0.
I20260929 06:33:08.962971 953 leader_election.cc:302] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cf25092ceba4f5e837cc4b30edd5171, fc1bb905203645329e9f7d38961e635d; no voters:
I20260929 06:33:08.963048 886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14f1a64fe524d8b94192e201151cc31" candidate_uuid: "fc1bb905203645329e9f7d38961e635d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" is_pre_election: true
I20260929 06:33:08.963090 1082 raft_consensus.cc:2781] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 06:33:08.963138 1082 raft_consensus.cc:482] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 06:33:08.963160 1082 raft_consensus.cc:3037] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.963199 886 raft_consensus.cc:2454] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1bb905203645329e9f7d38961e635d in term 0.
I20260929 06:33:08.963914 1082 raft_consensus.cc:504] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.964049 1082 leader_election.cc:288] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [CANDIDATE]: Term 1 election: Requested vote from peers 4df1cfaa6f5c4de2aa1b8686c4967c78 (127.13.15.130:34513), 2cf25092ceba4f5e837cc4b30edd5171 (127.13.15.129:34239)
I20260929 06:33:08.964185 886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14f1a64fe524d8b94192e201151cc31" candidate_uuid: "fc1bb905203645329e9f7d38961e635d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78"
I20260929 06:33:08.964269 886 raft_consensus.cc:3037] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.964344 755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14f1a64fe524d8b94192e201151cc31" candidate_uuid: "fc1bb905203645329e9f7d38961e635d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf25092ceba4f5e837cc4b30edd5171"
I20260929 06:33:08.964447 755 raft_consensus.cc:3037] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:08.965066 886 raft_consensus.cc:2454] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1bb905203645329e9f7d38961e635d in term 1.
I20260929 06:33:08.965205 755 raft_consensus.cc:2454] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1bb905203645329e9f7d38961e635d in term 1.
I20260929 06:33:08.965291 952 leader_election.cc:302] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df1cfaa6f5c4de2aa1b8686c4967c78, fc1bb905203645329e9f7d38961e635d; no voters:
I20260929 06:33:08.965395 1082 raft_consensus.cc:2781] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:08.965561 1082 raft_consensus.cc:686] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 1 LEADER]: Becoming Leader. State: Replica: fc1bb905203645329e9f7d38961e635d, State: Running, Role: LEADER
I20260929 06:33:08.965765 1082 consensus_queue.cc:237] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } }
I20260929 06:33:08.966730 613 catalog_manager.cc:5495] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1bb905203645329e9f7d38961e635d (127.13.15.131). New cstate: current_term: 1 leader_uuid: "fc1bb905203645329e9f7d38961e635d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1bb905203645329e9f7d38961e635d" member_type: VOTER last_known_addr { host: "127.13.15.131" port: 40485 } health_report { overall_health: HEALTHY } } }
I20260929 06:33:08.971992 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.969561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.13.15.131:40485)
0929 06:33:08.969598 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:08.969599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:08.969607 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:08.969689 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:08.969690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:08.969705 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:08.969771 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:08.969893 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.970421 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.971194 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:08.971388 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:08.971471 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:08.971640 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:08.971738 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:08.971842 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:08.971879 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 06:33:08.971894 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 06:33:08.973179 1064 tablet.cc:2369] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:08.973649 755 raft_consensus.cc:1261] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171 [term 1 FOLLOWER]: Refusing update from remote peer fc1bb905203645329e9f7d38961e635d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:08.973655 886 raft_consensus.cc:1261] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 1 FOLLOWER]: Refusing update from remote peer fc1bb905203645329e9f7d38961e635d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:08.973977 1082 consensus_queue.cc:1035] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df1cfaa6f5c4de2aa1b8686c4967c78" member_type: VOTER last_known_addr { host: "127.13.15.130" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:08.974124 1087 consensus_queue.cc:1035] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cf25092ceba4f5e837cc4b30edd5171" member_type: VOTER last_known_addr { host: "127.13.15.129" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:08.976956 1090 mvcc.cc:205] Tried to move back new op lower bound from 7334558060433395712 to 7334558060404600832. Current Snapshot: MvccSnapshot[applied={T|T < 7334558060433395712 or (T in {7334558060433395712})}]
I20260929 06:33:08.977065 1091 mvcc.cc:205] Tried to move back new op lower bound from 7334558060433395712 to 7334558060404600832. Current Snapshot: MvccSnapshot[applied={T|T < 7334558060433395712}]
W20260929 06:33:08.977560 802 tablet.cc:2369] T b14f1a64fe524d8b94192e201151cc31 P 2cf25092ceba4f5e837cc4b30edd5171: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 06:33:08.978493 1093 mvcc.cc:205] Tried to move back new op lower bound from 7334558060433395712 to 7334558060404600832. Current Snapshot: MvccSnapshot[applied={T|T < 7334558060433395712}]
I20260929 06:33:08.978469 612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:08.978571 612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 06:33:08.979313 612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33050646fa9347eb8064284b73982cd6: Sending DeleteTablet for 3 replicas of tablet b14f1a64fe524d8b94192e201151cc31
I20260929 06:33:08.979628 735 tablet_service.cc:1514] Processing DeleteTablet for tablet b14f1a64fe524d8b94192e201151cc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='slave'} at 127.0.0.1:35124
I20260929 06:33:08.979633 866 tablet_service.cc:1514] Processing DeleteTablet for tablet b14f1a64fe524d8b94192e201151cc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='slave'} at 127.0.0.1:38292
I20260929 06:33:08.979733 997 tablet_service.cc:1514] Processing DeleteTablet for tablet b14f1a64fe524d8b94192e201151cc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 06:33:08 UTC) from {username='slave'} at 127.0.0.1:60962
I20260929 06:33:08.979900 1101 tablet_replica.cc:330] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: stopping tablet replica
I20260929 06:33:08.980052 1102 tablet_replica.cc:330] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: stopping tablet replica
I20260929 06:33:08.980058 1101 raft_consensus.cc:2229] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 06:33:08.980096 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 673
I20260929 06:33:08.980235 1101 raft_consensus.cc:2258] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:08.980149 1102 raft_consensus.cc:2229] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 1 LEADER]: Raft consensus shutting down.
I20260929 06:33:08.980793 1102 raft_consensus.cc:2258] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 06:33:08.981259 1102 ts_tablet_manager.cc:1905] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:08.981410 1101 ts_tablet_manager.cc:1905] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 06:33:08.983120 1101 ts_tablet_manager.cc:1918] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:08.983172 1102 ts_tablet_manager.cc:1918] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 06:33:08.983242 1102 log.cc:1196] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-2/wal/wals/b14f1a64fe524d8b94192e201151cc31
I20260929 06:33:08.983321 1101 log.cc:1196] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Deleting WAL directory at /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790663323716959-13374-0/minicluster-data/ts-1/wal/wals/b14f1a64fe524d8b94192e201151cc31
I20260929 06:33:08.983516 1102 ts_tablet_manager.cc:1939] T b14f1a64fe524d8b94192e201151cc31 P fc1bb905203645329e9f7d38961e635d: Deleting consensus metadata
I20260929 06:33:08.983707 1101 ts_tablet_manager.cc:1939] T b14f1a64fe524d8b94192e201151cc31 P 4df1cfaa6f5c4de2aa1b8686c4967c78: Deleting consensus metadata
I20260929 06:33:08.983953 600 catalog_manager.cc:4841] TS fc1bb905203645329e9f7d38961e635d (127.13.15.131:40485): tablet b14f1a64fe524d8b94192e201151cc31 (table test-table [id=03615cfab18e40949fb4473f0d3c21e5]) successfully deleted
I20260929 06:33:08.984191 599 catalog_manager.cc:4841] TS 4df1cfaa6f5c4de2aa1b8686c4967c78 (127.13.15.130:34513): tablet b14f1a64fe524d8b94192e201151cc31 (table test-table [id=03615cfab18e40949fb4473f0d3c21e5]) successfully deleted
W20260929 06:33:08.986399 600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.15.129:34239 (error 108)
I20260929 06:33:08.986907 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:08.986757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.129:34239 (local address 127.0.0.1:35126)
0929 06:33:08.986817 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:08.986872 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.129:34239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 06:33:08.986968 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 804
W20260929 06:33:08.989430 600 catalog_manager.cc:4584] TS 2cf25092ceba4f5e837cc4b30edd5171 (127.13.15.129:34239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b14f1a64fe524d8b94192e201151cc31: Network error: Client connection negotiation failed: client connection to 127.13.15.129:34239: connect: Connection refused (error 111)
I20260929 06:33:08.992697 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 935
I20260929 06:33:08.997867 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 582
2026-09-29T06:33:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (844 ms)
[----------] 6 tests from EncryptionPolicyTest (8364 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:09 dist-test-slave-sm0p krb5kdc[1106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:09 dist-test-slave-sm0p krb5kdc[1106](info): set up 2 sockets
Sep 29 06:33:09 dist-test-slave-sm0p krb5kdc[1106](info): commencing operation
krb5kdc: starting...
W20260929 06:33:10.022034 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:10.136376 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:10.217154 1119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:10.217417 1119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:10.217454 1119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:10.218664 1119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:10.218740 1119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:10.218765 1119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:10.218787 1119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:10.218814 1119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:10.220322 1119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:10.220662 1119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:10.220893 1119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:10.225759 1125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:10.225777 1127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:10.225975 1119 server_base.cc:1034] running on GCE node
W20260929 06:33:10.225795 1124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:10.226302 1119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:10.226600 1119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:10.227766 1119 hybrid_clock.cc:648] HybridClock initialized: now 1790663590227758 us; error 28 us; skew 500 ppm
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:10.230410 1119 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:10.230860 1119 webserver.cc:460] Webserver started at http://127.0.0.1:34873/ using document root <none> and password file <none>
I20260929 06:33:10.231065 1119 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:10.231105 1119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:10.231197 1119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:10.232198 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "9db558365dbb473d8ec5af6101d8f54e"
format_stamp: "Formatted at 2026-09-29 06:33:10 on dist-test-slave-sm0p"
server_key: "f8bb243f5141ee36349faa4a1de3165b"
server_key_iv: "0e529d521e156e9f04d18797f8454e3b"
server_key_version: "encryptionkey@0"
I20260929 06:33:10.232508 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "9db558365dbb473d8ec5af6101d8f54e"
format_stamp: "Formatted at 2026-09-29 06:33:10 on dist-test-slave-sm0p"
server_key: "f8bb243f5141ee36349faa4a1de3165b"
server_key_iv: "0e529d521e156e9f04d18797f8454e3b"
server_key_version: "encryptionkey@0"
I20260929 06:33:10.233764 1119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:10.234506 1134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:10.234702 1119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:10.234793 1119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "9db558365dbb473d8ec5af6101d8f54e"
format_stamp: "Formatted at 2026-09-29 06:33:10 on dist-test-slave-sm0p"
server_key: "f8bb243f5141ee36349faa4a1de3165b"
server_key_iv: "0e529d521e156e9f04d18797f8454e3b"
server_key_version: "encryptionkey@0"
I20260929 06:33:10.234848 1119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:10.255051 1119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:10.256870 1119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:10.256979 1119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:10.260862 1119 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37465
I20260929 06:33:10.260947 1186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37465 every 8 connection(s)
I20260929 06:33:10.261296 1119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:10.262391 1187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:10.264689 1187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e: Bootstrap starting.
I20260929 06:33:10.265291 1187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:10.265579 1187 log.cc:824] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:10.266301 1187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e: No bootstrap required, opened a new log
I20260929 06:33:10.267901 1187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 06:33:10.268070 1187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:10.268117 1187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db558365dbb473d8ec5af6101d8f54e, State: Initialized, Role: FOLLOWER
I20260929 06:33:10.268285 1187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 06:33:10.268369 1187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:10.268411 1187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:10.268461 1187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:10.269006 1187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 06:33:10.269124 1187 leader_election.cc:302] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9db558365dbb473d8ec5af6101d8f54e; no voters:
I20260929 06:33:10.269306 1187 leader_election.cc:288] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:10.269347 1190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:10.269526 1190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [term 1 LEADER]: Becoming Leader. State: Replica: 9db558365dbb473d8ec5af6101d8f54e, State: Running, Role: LEADER
I20260929 06:33:10.269579 1187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:10.269618 1190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } }
I20260929 06:33:10.270016 1192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9db558365dbb473d8ec5af6101d8f54e. Latest consensus state: current_term: 1 leader_uuid: "9db558365dbb473d8ec5af6101d8f54e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } } }
I20260929 06:33:10.270102 1192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:10.270234 1191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9db558365dbb473d8ec5af6101d8f54e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db558365dbb473d8ec5af6101d8f54e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37465 } } }
I20260929 06:33:10.270303 1191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:10.270754 1198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:10.271250 1198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:10.271461 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1119
I20260929 06:33:10.271591 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:10.271796 13374 external_mini_cluster.cc:1428] Setting key d2910e157b6bc41c1eb5806037c93c71
I20260929 06:33:10.273062 1198 catalog_manager.cc:1347] Generated new cluster ID: 1951bb621f7f44798c09ad98637bc00f
I20260929 06:33:10.273114 1198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 06:33:10.283838 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:10.272266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48591 (local address 127.0.0.1:37465)
0929 06:33:10.272446 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:10.272455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:10.272646 (+ 191us) server_negotiation.cc:407] Connection header received
0929 06:33:10.272799 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:10.272806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:10.272866 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:10.273855 (+ 989us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:10.273950 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:10.275410 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:10.276451 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:10.277091 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:10.277204 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:10.280016 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:10.280020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:10.280020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:10.280033 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:10.282002 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:10.282511 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:10.282514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:10.282514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:10.282625 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:10.282878 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:10.282880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:10.282880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:10.283173 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:10.283322 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:10.283481 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 06:33:10.283579 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260929 06:33:10.284847 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:10.291137 1198 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:10.291847 1198 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:10.293866 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:10.286467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39391 (local address 127.0.0.1:37465)
0929 06:33:10.286499 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:10.286500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:10.286508 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:10.286570 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:10.286570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:10.286579 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:10.286627 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:10.287235 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:10.287965 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:10.288995 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:10.289300 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:10.289366 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:10.290869 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:10.290870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:10.290871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:10.290878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:10.292565 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:10.292990 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:10.292991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:10.292991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:10.293045 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:10.293270 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:10.293270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:10.293271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:10.293387 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:10.293473 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:10.293629 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 06:33:10.293688 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:33:10.297960 1198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9db558365dbb473d8ec5af6101d8f54e: Generated new TSK 0
I20260929 06:33:10.298194 1198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:10.300511 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1119
W20260929 06:33:10.305009 1104 connection.cc:535] client connection to 127.0.0.1:37465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37465): Connection reset by peer (error 104)
2026-09-29T06:33:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1218](info): set up 2 sockets
Sep 29 06:33:10 dist-test-slave-sm0p krb5kdc[1218](info): commencing operation
krb5kdc: starting...
W20260929 06:33:11.322217 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:11 dist-test-slave-sm0p krb5kdc[1218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:11.440502 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:11.522270 1231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:11.522465 1231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:11.522487 1231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:11.523661 1231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:11.523705 1231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:11.523720 1231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:11.523758 1231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:11.523787 1231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:11.525244 1231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:11.525540 1231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:11.525761 1231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:11.531095 1236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:11.531085 1239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:11.531196 1237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:11.531296 1231 server_base.cc:1034] running on GCE node
I20260929 06:33:11.531648 1231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:11.532030 1231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:11.533210 1231 hybrid_clock.cc:648] HybridClock initialized: now 1790663591533198 us; error 32 us; skew 500 ppm
Sep 29 06:33:11 dist-test-slave-sm0p krb5kdc[1218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:11.543349 1231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:11.543754 1231 webserver.cc:460] Webserver started at http://127.0.0.1:38589/ using document root <none> and password file <none>
I20260929 06:33:11.543954 1231 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:11.543996 1231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:11.544085 1231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:11.545095 1231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd"
format_stamp: "Formatted at 2026-09-29 06:33:11 on dist-test-slave-sm0p"
server_key: "cb34948c13ee6876618cca1a3977d30a"
server_key_iv: "b2679e3145d7b8ef03c53c96e9484396"
server_key_version: "encryptionkey@0"
I20260929 06:33:11.545388 1231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd"
format_stamp: "Formatted at 2026-09-29 06:33:11 on dist-test-slave-sm0p"
server_key: "cb34948c13ee6876618cca1a3977d30a"
server_key_iv: "b2679e3145d7b8ef03c53c96e9484396"
server_key_version: "encryptionkey@0"
I20260929 06:33:11.546682 1231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:11.547353 1246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:11.547562 1231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:11.547649 1231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd"
format_stamp: "Formatted at 2026-09-29 06:33:11 on dist-test-slave-sm0p"
server_key: "cb34948c13ee6876618cca1a3977d30a"
server_key_iv: "b2679e3145d7b8ef03c53c96e9484396"
server_key_version: "encryptionkey@0"
I20260929 06:33:11.547719 1231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:11.559458 1231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:11.561195 1231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:11.561347 1231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:11.565285 1231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36147
I20260929 06:33:11.565316 1298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36147 every 8 connection(s)
I20260929 06:33:11.565658 1231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:11.566587 1299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:11.568298 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1231
I20260929 06:33:11.568421 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:11.568612 13374 external_mini_cluster.cc:1428] Setting key e11ebea639c4425c4ba6e030135df920
I20260929 06:33:11.569078 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd: Bootstrap starting.
I20260929 06:33:11.569664 1299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:11.569978 1299 log.cc:824] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:11.570694 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd: No bootstrap required, opened a new log
I20260929 06:33:11.572953 1299 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260929 06:33:11.573139 1299 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:11.573189 1299 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdec6e4365a44e1eaf12a1e7c7c060dd, State: Initialized, Role: FOLLOWER
I20260929 06:33:11.573352 1299 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260929 06:33:11.573443 1299 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:11.573482 1299 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:11.573537 1299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:11.574239 1299 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260929 06:33:11.574261 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:11.569101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52447 (local address 127.0.0.1:36147)
0929 06:33:11.569268 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:11.569277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:11.569311 (+ 34us) server_negotiation.cc:407] Connection header received
0929 06:33:11.569776 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:11.569783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:11.569836 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:11.569945 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:11.570194 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:11.571207 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:11.572295 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:11.572696 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:11.572771 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:11.573532 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:11.573537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:11.573537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:11.573725 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:11.573753 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:11.573763 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:11.573825 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:11.573995 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 06:33:11.574025 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260929 06:33:11.574373 1299 leader_election.cc:302] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdec6e4365a44e1eaf12a1e7c7c060dd; no voters:
I20260929 06:33:11.574550 1299 leader_election.cc:288] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:11.574579 1304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:11.574726 1304 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [term 1 LEADER]: Becoming Leader. State: Replica: fdec6e4365a44e1eaf12a1e7c7c060dd, State: Running, Role: LEADER
I20260929 06:33:11.574843 1299 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:11.575032 1304 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260929 06:33:11.575217 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:11.575350 1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdec6e4365a44e1eaf12a1e7c7c060dd. Latest consensus state: current_term: 1 leader_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } } }
I20260929 06:33:11.575425 1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:11.575639 1313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:11.576123 1313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:11.577054 1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdec6e4365a44e1eaf12a1e7c7c060dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } } }
I20260929 06:33:11.577332 1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:11.578477 1313 catalog_manager.cc:1347] Generated new cluster ID: 682cf83db51e42669699b90e341c5e19
I20260929 06:33:11.578531 1313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:11.580787 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:11.577184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59261 (local address 127.0.0.1:36147)
0929 06:33:11.577217 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:11.577219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:11.577529 (+ 310us) server_negotiation.cc:407] Connection header received
0929 06:33:11.577572 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:11.577573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:11.577582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:11.577637 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:11.577769 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:11.578342 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:11.579392 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:11.579687 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:11.579733 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:11.580519 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:11.580521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:11.580521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:11.580525 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:11.580536 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:11.580540 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:11.580583 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:11.580711 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 06:33:11.580720 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:33:11.593014 1313 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:11.593704 1313 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:11.599692 1313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdec6e4365a44e1eaf12a1e7c7c060dd: Generated new TSK 0
I20260929 06:33:11.599874 1313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:11.601647 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1231
W20260929 06:33:11.605988 1216 connection.cc:535] client connection to 127.0.0.1:36147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36147): Connection reset by peer (error 104)
2026-09-29T06:33:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:11 dist-test-slave-sm0p krb5kdc[1330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:11 dist-test-slave-sm0p krb5kdc[1330](info): set up 2 sockets
Sep 29 06:33:11 dist-test-slave-sm0p krb5kdc[1330](info): commencing operation
krb5kdc: starting...
W20260929 06:33:12.630033 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:12 dist-test-slave-sm0p krb5kdc[1330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:12.754040 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:12.831106 1343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:12.831300 1343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:12.831321 1343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:12.832362 1343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:12.832394 1343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:12.832408 1343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:12.832420 1343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:12.832432 1343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:12.833771 1343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:12.833956 1343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:12.834120 1343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:12.838930 1343 server_base.cc:1034] running on GCE node
W20260929 06:33:12.838903 1351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:12.838898 1349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:12.838905 1348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:12.839417 1343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:12.839704 1343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:12.840862 1343 hybrid_clock.cc:648] HybridClock initialized: now 1790663592840854 us; error 37 us; skew 500 ppm
Sep 29 06:33:12 dist-test-slave-sm0p krb5kdc[1330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:12.843317 1343 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:12.843786 1343 webserver.cc:460] Webserver started at http://127.0.0.1:33475/ using document root <none> and password file <none>
I20260929 06:33:12.843999 1343 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:12.844066 1343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:12.844214 1343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:12.845219 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "2ef066bf87bf4353bf68a60bbbd54d1d"
format_stamp: "Formatted at 2026-09-29 06:33:12 on dist-test-slave-sm0p"
server_key: "1b7afa52e56fa36f188baddc433f2caa"
server_key_iv: "dbf6132206fd464b75e088d6499e7b49"
server_key_version: "encryptionkey@0"
I20260929 06:33:12.845574 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "2ef066bf87bf4353bf68a60bbbd54d1d"
format_stamp: "Formatted at 2026-09-29 06:33:12 on dist-test-slave-sm0p"
server_key: "1b7afa52e56fa36f188baddc433f2caa"
server_key_iv: "dbf6132206fd464b75e088d6499e7b49"
server_key_version: "encryptionkey@0"
I20260929 06:33:12.846932 1343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:12.847643 1358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:12.847862 1343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:12.847983 1343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "2ef066bf87bf4353bf68a60bbbd54d1d"
format_stamp: "Formatted at 2026-09-29 06:33:12 on dist-test-slave-sm0p"
server_key: "1b7afa52e56fa36f188baddc433f2caa"
server_key_iv: "dbf6132206fd464b75e088d6499e7b49"
server_key_version: "encryptionkey@0"
I20260929 06:33:12.848059 1343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:12.851320 1343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:12.853158 1343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:12.853309 1343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:12.857383 1343 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43033
I20260929 06:33:12.857440 1410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43033 every 8 connection(s)
I20260929 06:33:12.857743 1343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:12.858498 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1343
I20260929 06:33:12.858652 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:12.858871 13374 external_mini_cluster.cc:1428] Setting key 3150d078cf45894532a187f669150680
I20260929 06:33:12.858871 1411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:12.861126 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:12.859377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37401 (local address 127.0.0.1:43033)
0929 06:33:12.859572 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:12.859578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:12.859601 (+ 23us) server_negotiation.cc:407] Connection header received
0929 06:33:12.859997 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:12.860003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:12.860025 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:12.860128 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:12.860716 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:12.860718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:12.860719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:12.860722 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:12.860752 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:12.860761 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:12.860811 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:12.860893 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 06:33:12.860921 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260929 06:33:12.862085 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:12.862689 1411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d: Bootstrap starting.
I20260929 06:33:12.863483 1411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:12.863890 1411 log.cc:824] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:12.864701 1411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d: No bootstrap required, opened a new log
I20260929 06:33:12.864821 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:12.863805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48797 (local address 127.0.0.1:43033)
0929 06:33:12.863842 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:12.863844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:12.863932 (+ 88us) server_negotiation.cc:407] Connection header received
0929 06:33:12.863991 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:12.863992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:12.863997 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:12.864046 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:12.864595 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:12.864597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:12.864597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:12.864600 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:12.864611 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:12.864614 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:12.864669 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:12.864726 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:33:12.864734 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:33:12.866801 1411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260929 06:33:12.866935 1411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:12.866961 1411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef066bf87bf4353bf68a60bbbd54d1d, State: Initialized, Role: FOLLOWER
I20260929 06:33:12.867040 1411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260929 06:33:12.867100 1411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:12.867122 1411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:12.867161 1411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:12.867650 1411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260929 06:33:12.867733 1411 leader_election.cc:302] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ef066bf87bf4353bf68a60bbbd54d1d; no voters:
I20260929 06:33:12.867848 1411 leader_election.cc:288] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:12.867909 1421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:12.868052 1421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [term 1 LEADER]: Becoming Leader. State: Replica: 2ef066bf87bf4353bf68a60bbbd54d1d, State: Running, Role: LEADER
I20260929 06:33:12.868095 1411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:12.868130 1421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } }
I20260929 06:33:12.868487 1423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ef066bf87bf4353bf68a60bbbd54d1d. Latest consensus state: current_term: 1 leader_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } } }
I20260929 06:33:12.868530 1422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef066bf87bf4353bf68a60bbbd54d1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43033 } } }
I20260929 06:33:12.868647 1423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:12.868670 1422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:12.869031 1430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:12.869498 1430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:12.871012 1430 catalog_manager.cc:1347] Generated new cluster ID: e7acecd9002b459f8348819388ffebac
I20260929 06:33:12.871071 1430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:12.881708 1430 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:12.881860 1430 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:12.889734 1430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ef066bf87bf4353bf68a60bbbd54d1d: Generated new TSK 0
I20260929 06:33:12.889915 1430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:12.896737 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1343
2026-09-29T06:33:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:12 dist-test-slave-sm0p krb5kdc[1442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:12 dist-test-slave-sm0p krb5kdc[1442](info): set up 2 sockets
Sep 29 06:33:12 dist-test-slave-sm0p krb5kdc[1442](info): commencing operation
krb5kdc: starting...
W20260929 06:33:13.906107 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:13 dist-test-slave-sm0p krb5kdc[1442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:14.016618 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:14.094304 1455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:14.094496 1455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:14.094516 1455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:14.095695 1455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:14.095752 1455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:14.095767 1455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:14.095779 1455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:14.095791 1455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:14.097219 1455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:14.097453 1455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:14.097669 1455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:14.102339 1463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:14.102347 1460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:14.102473 1455 server_base.cc:1034] running on GCE node
W20260929 06:33:14.102347 1461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:14.102772 1455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:14.103034 1455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:14.104167 1455 hybrid_clock.cc:648] HybridClock initialized: now 1790663594104159 us; error 29 us; skew 500 ppm
Sep 29 06:33:14 dist-test-slave-sm0p krb5kdc[1442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:14.106518 1455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:14.106983 1455 webserver.cc:460] Webserver started at http://127.0.0.1:46803/ using document root <none> and password file <none>
I20260929 06:33:14.107206 1455 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:14.107249 1455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:14.107412 1455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:14.108424 1455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "4daa3f8c474845c19a75947b2a18eb81"
format_stamp: "Formatted at 2026-09-29 06:33:14 on dist-test-slave-sm0p"
server_key: "3b91c14facd8b9dc43c33d5f8dd83f53"
server_key_iv: "7b74093e62511f17543b7f576290254d"
server_key_version: "encryptionkey@0"
I20260929 06:33:14.108770 1455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "4daa3f8c474845c19a75947b2a18eb81"
format_stamp: "Formatted at 2026-09-29 06:33:14 on dist-test-slave-sm0p"
server_key: "3b91c14facd8b9dc43c33d5f8dd83f53"
server_key_iv: "7b74093e62511f17543b7f576290254d"
server_key_version: "encryptionkey@0"
I20260929 06:33:14.110031 1455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:33:14.110750 1470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:14.110929 1455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:14.111012 1455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "4daa3f8c474845c19a75947b2a18eb81"
format_stamp: "Formatted at 2026-09-29 06:33:14 on dist-test-slave-sm0p"
server_key: "3b91c14facd8b9dc43c33d5f8dd83f53"
server_key_iv: "7b74093e62511f17543b7f576290254d"
server_key_version: "encryptionkey@0"
I20260929 06:33:14.111074 1455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:14.128402 1455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:14.129994 1455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:14.130136 1455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:14.134196 1455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39609
I20260929 06:33:14.134227 1522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39609 every 8 connection(s)
I20260929 06:33:14.134565 1455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:14.135375 1523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:14.137485 1523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81: Bootstrap starting.
I20260929 06:33:14.138038 1523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:14.138330 1523 log.cc:824] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:14.138979 1523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81: No bootstrap required, opened a new log
I20260929 06:33:14.140456 1523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } }
I20260929 06:33:14.140585 1523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:14.140606 1523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4daa3f8c474845c19a75947b2a18eb81, State: Initialized, Role: FOLLOWER
I20260929 06:33:14.140797 1523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } }
I20260929 06:33:14.140853 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1455
I20260929 06:33:14.140901 1523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:14.140939 1523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:14.140965 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:14.140995 1523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:14.141150 13374 external_mini_cluster.cc:1428] Setting key 11bbeb6586f293f669e91775a7f21579
I20260929 06:33:14.141691 1523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } }
I20260929 06:33:14.141798 1523 leader_election.cc:302] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4daa3f8c474845c19a75947b2a18eb81; no voters:
I20260929 06:33:14.141976 1523 leader_election.cc:288] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:14.142020 1528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:14.142231 1528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [term 1 LEADER]: Becoming Leader. State: Replica: 4daa3f8c474845c19a75947b2a18eb81, State: Running, Role: LEADER
I20260929 06:33:14.142261 1523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:14.142331 1528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } }
I20260929 06:33:14.142657 1528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4daa3f8c474845c19a75947b2a18eb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } } }
I20260929 06:33:14.142776 1528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:14.142998 1529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4daa3f8c474845c19a75947b2a18eb81. Latest consensus state: current_term: 1 leader_uuid: "4daa3f8c474845c19a75947b2a18eb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4daa3f8c474845c19a75947b2a18eb81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39609 } } }
I20260929 06:33:14.143090 1537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:14.143671 1537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:14.143164 1529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:14.145143 1537 catalog_manager.cc:1347] Generated new cluster ID: ca82243c3287412386006fa2020769d6
I20260929 06:33:14.145186 1537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:33:14 dist-test-slave-sm0p krb5kdc[1442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 06:33:14.151052 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:14.141606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33287 (local address 127.0.0.1:39609)
0929 06:33:14.141777 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:14.141787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:14.141818 (+ 31us) server_negotiation.cc:407] Connection header received
0929 06:33:14.141933 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:14.141940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:14.141996 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:14.142149 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:14.142953 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:14.144389 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:14.145202 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:14.145612 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:14.145735 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:14.147988 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:14.147992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:14.147993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:14.148003 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:14.149564 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:14.149941 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:14.149944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:14.149944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:14.150042 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:14.150275 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:14.150276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:14.150276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:14.150494 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:14.150659 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:14.150749 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 06:33:14.150897 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260929 06:33:14.151916 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:14.160087 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:14.153224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34985 (local address 127.0.0.1:39609)
0929 06:33:14.153265 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:14.153267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:14.154326 (+ 1059us) server_negotiation.cc:407] Connection header received
0929 06:33:14.154402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:14.154419 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:14.154429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:14.154483 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:14.155004 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:14.155444 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:14.156138 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:14.156344 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:14.156420 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:14.157511 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:14.157512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:14.157513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:14.157518 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:14.158968 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:14.159320 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:14.159321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:14.159321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:14.159373 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:14.159579 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:14.159580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:14.159580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:14.159698 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:14.159814 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:14.159916 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 06:33:14.159988 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 06:33:14.165692 1537 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:14.166391 1537 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:14.184638 1537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4daa3f8c474845c19a75947b2a18eb81: Generated new TSK 0
I20260929 06:33:14.184814 1537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:14.187606 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1455
2026-09-29T06:33:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:14 dist-test-slave-sm0p krb5kdc[1554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:14 dist-test-slave-sm0p krb5kdc[1554](info): set up 2 sockets
Sep 29 06:33:14 dist-test-slave-sm0p krb5kdc[1554](info): commencing operation
krb5kdc: starting...
W20260929 06:33:15.182068 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:15 dist-test-slave-sm0p krb5kdc[1554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:15.292909 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:15.368844 1567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:15.369023 1567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:15.369045 1567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:15.370132 1567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:15.370186 1567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:15.370201 1567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:15.370213 1567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:15.370225 1567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:15.371604 1567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:15.371770 1567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:15.371999 1567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:15.376757 1572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:15.376763 1573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:15.376796 1567 server_base.cc:1034] running on GCE node
W20260929 06:33:15.376763 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:15.377218 1567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:15.377444 1567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:15.378621 1567 hybrid_clock.cc:648] HybridClock initialized: now 1790663595378616 us; error 50 us; skew 500 ppm
Sep 29 06:33:15 dist-test-slave-sm0p krb5kdc[1554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:15.381065 1567 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:15.381413 1567 webserver.cc:460] Webserver started at http://127.0.0.1:33673/ using document root <none> and password file <none>
I20260929 06:33:15.381590 1567 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:15.381629 1567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:15.381711 1567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:15.382685 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "407d8bf2c5dd4354b6abadb1501c4e5f"
format_stamp: "Formatted at 2026-09-29 06:33:15 on dist-test-slave-sm0p"
server_key: "26edb716e0f0c78e84001d25946aa2a3"
server_key_iv: "b567ddeb3cb08b4249b91a9ee903bb20"
server_key_version: "encryptionkey@0"
I20260929 06:33:15.382977 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "407d8bf2c5dd4354b6abadb1501c4e5f"
format_stamp: "Formatted at 2026-09-29 06:33:15 on dist-test-slave-sm0p"
server_key: "26edb716e0f0c78e84001d25946aa2a3"
server_key_iv: "b567ddeb3cb08b4249b91a9ee903bb20"
server_key_version: "encryptionkey@0"
I20260929 06:33:15.384193 1567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:33:15.385000 1582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:15.385180 1567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:15.385281 1567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "407d8bf2c5dd4354b6abadb1501c4e5f"
format_stamp: "Formatted at 2026-09-29 06:33:15 on dist-test-slave-sm0p"
server_key: "26edb716e0f0c78e84001d25946aa2a3"
server_key_iv: "b567ddeb3cb08b4249b91a9ee903bb20"
server_key_version: "encryptionkey@0"
I20260929 06:33:15.385355 1567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:15.396941 1567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:15.399236 1567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:15.399403 1567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:15.403496 1567 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46723
I20260929 06:33:15.403559 1634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46723 every 8 connection(s)
I20260929 06:33:15.403884 1567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:15.404675 1635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:15.406967 1635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f: Bootstrap starting.
I20260929 06:33:15.407181 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1567
I20260929 06:33:15.407330 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:15.407523 13374 external_mini_cluster.cc:1428] Setting key 0cc79d3ccadaeda4ae2a370fbe408889
I20260929 06:33:15.407707 1635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:15.408064 1635 log.cc:824] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:15.408785 1635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f: No bootstrap required, opened a new log
I20260929 06:33:15.410607 1635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260929 06:33:15.410759 1635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:15.410805 1635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407d8bf2c5dd4354b6abadb1501c4e5f, State: Initialized, Role: FOLLOWER
I20260929 06:33:15.410976 1635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260929 06:33:15.411089 1635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:15.411134 1635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:15.411178 1635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:15.411693 1635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260929 06:33:15.411818 1635 leader_election.cc:302] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 407d8bf2c5dd4354b6abadb1501c4e5f; no voters:
I20260929 06:33:15.412022 1635 leader_election.cc:288] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:15.412071 1640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:15.412292 1635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:15.412493 1640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [term 1 LEADER]: Becoming Leader. State: Replica: 407d8bf2c5dd4354b6abadb1501c4e5f, State: Running, Role: LEADER
I20260929 06:33:15.412642 1640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260929 06:33:15.413065 1641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } } }
I20260929 06:33:15.413146 1641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:15.413295 1642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 407d8bf2c5dd4354b6abadb1501c4e5f. Latest consensus state: current_term: 1 leader_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "407d8bf2c5dd4354b6abadb1501c4e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } } }
I20260929 06:33:15.413336 1649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:15.413398 1642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:15.413300 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:15.408015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57553 (local address 127.0.0.1:46723)
0929 06:33:15.408208 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:15.408218 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:15.408256 (+ 38us) server_negotiation.cc:407] Connection header received
0929 06:33:15.408800 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:15.408808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:15.408896 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:15.409052 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:15.409293 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:15.410214 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:15.411089 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:15.411567 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:15.411688 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:15.412413 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:15.412417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:15.412418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:15.412622 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:15.412659 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:15.412673 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:15.412998 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:15.413030 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 06:33:15.413097 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260929 06:33:15.413967 1649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:15.414479 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:15.415660 1649 catalog_manager.cc:1347] Generated new cluster ID: ca666c59b4ab4e9f8722a119e4a1e686
I20260929 06:33:15.415710 1649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:15.418747 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:15.416009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59555 (local address 127.0.0.1:46723)
0929 06:33:15.416042 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:15.416044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:15.416187 (+ 143us) server_negotiation.cc:407] Connection header received
0929 06:33:15.416255 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:15.416256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:15.416265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:15.416318 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:15.416450 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:15.416904 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:15.417587 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:15.417795 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:15.417874 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:15.418500 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:15.418501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:15.418501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:15.418505 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:15.418517 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:15.418520 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:15.418568 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:15.418622 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 06:33:15.418640 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:33:15.423663 1649 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:15.424309 1649 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:15.434180 1649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 407d8bf2c5dd4354b6abadb1501c4e5f: Generated new TSK 0
I20260929 06:33:15.434369 1649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:15.438023 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1567
2026-09-29T06:33:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1250 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:15 dist-test-slave-sm0p krb5kdc[1666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:15 dist-test-slave-sm0p krb5kdc[1666](info): set up 2 sockets
Sep 29 06:33:15 dist-test-slave-sm0p krb5kdc[1666](info): commencing operation
krb5kdc: starting...
W20260929 06:33:16.450081 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:16 dist-test-slave-sm0p krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:16.562981 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43169
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:16.641213 1679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:16.641422 1679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:16.641458 1679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:16.642702 1679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:16.642766 1679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:16.642789 1679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:16.642828 1679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:16.642875 1679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:16.644309 1679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:16.644572 1679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:16.644793 1679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:16.649457 1684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:16.649441 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:16.649441 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:16.649556 1679 server_base.cc:1034] running on GCE node
I20260929 06:33:16.649935 1679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:16.650259 1679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:16.651415 1679 hybrid_clock.cc:648] HybridClock initialized: now 1790663596651404 us; error 26 us; skew 500 ppm
Sep 29 06:33:16 dist-test-slave-sm0p krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:16.659036 1679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:16.659441 1679 webserver.cc:460] Webserver started at http://127.0.0.1:39927/ using document root <none> and password file <none>
I20260929 06:33:16.659665 1679 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:16.659730 1679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:16.659889 1679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:16.660897 1679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "692348c591534bb0b78bde8b3af43d7a"
format_stamp: "Formatted at 2026-09-29 06:33:16 on dist-test-slave-sm0p"
server_key: "8f2ef4d4714ecd4f9f1406cf5febdb09"
server_key_iv: "f94b702d7a0e040655277cf6db8edf81"
server_key_version: "encryptionkey@0"
I20260929 06:33:16.661240 1679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "692348c591534bb0b78bde8b3af43d7a"
format_stamp: "Formatted at 2026-09-29 06:33:16 on dist-test-slave-sm0p"
server_key: "8f2ef4d4714ecd4f9f1406cf5febdb09"
server_key_iv: "f94b702d7a0e040655277cf6db8edf81"
server_key_version: "encryptionkey@0"
I20260929 06:33:16.662528 1679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 06:33:16.663234 1694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:16.663414 1679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:16.663507 1679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "692348c591534bb0b78bde8b3af43d7a"
format_stamp: "Formatted at 2026-09-29 06:33:16 on dist-test-slave-sm0p"
server_key: "8f2ef4d4714ecd4f9f1406cf5febdb09"
server_key_iv: "f94b702d7a0e040655277cf6db8edf81"
server_key_version: "encryptionkey@0"
I20260929 06:33:16.663584 1679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:16.666739 1679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:16.668938 1679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:16.669098 1679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:16.673046 1679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43169
I20260929 06:33:16.673074 1746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43169 every 8 connection(s)
I20260929 06:33:16.673394 1679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:16.674211 1747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:16.676220 1747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a: Bootstrap starting.
I20260929 06:33:16.676698 1747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:16.676923 1747 log.cc:824] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:16.677477 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1679
I20260929 06:33:16.677536 1747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a: No bootstrap required, opened a new log
I20260929 06:33:16.677604 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:16.677772 13374 external_mini_cluster.cc:1428] Setting key a504defe5b64e765b53e2ce575c1f123
I20260929 06:33:16.680130 1747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } }
I20260929 06:33:16.680394 1747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:16.680496 1747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 692348c591534bb0b78bde8b3af43d7a, State: Initialized, Role: FOLLOWER
I20260929 06:33:16.680667 1747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } }
I20260929 06:33:16.680773 1747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:16.680812 1747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:16.680886 1747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:16.681320 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:16.678322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42415 (local address 127.0.0.1:43169)
0929 06:33:16.678523 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:16.678531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:16.678561 (+ 30us) server_negotiation.cc:407] Connection header received
0929 06:33:16.680066 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:16.680091 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:16.680126 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:16.680296 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:16.680868 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:16.680871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:16.680872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:16.680876 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:16.680916 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:16.680929 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:16.680996 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:16.681099 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 06:33:16.681153 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260929 06:33:16.681468 1747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } }
I20260929 06:33:16.681561 1747 leader_election.cc:302] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 692348c591534bb0b78bde8b3af43d7a; no voters:
I20260929 06:33:16.681702 1747 leader_election.cc:288] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:16.681731 1752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:16.681857 1752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [term 1 LEADER]: Becoming Leader. State: Replica: 692348c591534bb0b78bde8b3af43d7a, State: Running, Role: LEADER
I20260929 06:33:16.681952 1747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:16.682067 1752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } }
I20260929 06:33:16.682308 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:16.682458 1753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "692348c591534bb0b78bde8b3af43d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } } }
I20260929 06:33:16.682530 1753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:16.682653 1753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 692348c591534bb0b78bde8b3af43d7a. Latest consensus state: current_term: 1 leader_uuid: "692348c591534bb0b78bde8b3af43d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "692348c591534bb0b78bde8b3af43d7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43169 } } }
I20260929 06:33:16.682686 1753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:16.682720 1760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:16.683256 1760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:16.685149 1760 catalog_manager.cc:1347] Generated new cluster ID: 36d21eebc7c44e8ba5a8335f457a7d88
I20260929 06:33:16.685225 1760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:16.685317 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:16.684024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37299 (local address 127.0.0.1:43169)
0929 06:33:16.684083 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:16.684084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:16.684250 (+ 166us) server_negotiation.cc:407] Connection header received
0929 06:33:16.684323 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:16.684324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:16.684339 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:16.684396 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:16.684966 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:16.684968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:16.684968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:16.684971 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:16.684983 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:16.684988 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:16.685015 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:16.685114 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 06:33:16.685125 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:33:16.697655 1760 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:16.697796 1760 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:16.704753 1760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 692348c591534bb0b78bde8b3af43d7a: Generated new TSK 0
I20260929 06:33:16.704905 1760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:16.711576 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1679
2026-09-29T06:33:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:16 dist-test-slave-sm0p krb5kdc[1778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:16 dist-test-slave-sm0p krb5kdc[1778](info): set up 2 sockets
Sep 29 06:33:16 dist-test-slave-sm0p krb5kdc[1778](info): commencing operation
krb5kdc: starting...
W20260929 06:33:17.742051 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:17 dist-test-slave-sm0p krb5kdc[1778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:17.863080 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:17.942636 1791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:17.942898 1791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:17.942937 1791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:17.944118 1791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:17.944198 1791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:17.944223 1791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:17.944247 1791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:17.944275 1791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:17.945744 1791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:17.946040 1791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:17.946298 1791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:17.951009 1797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:17.951025 1796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:17.951020 1799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:17.951201 1791 server_base.cc:1034] running on GCE node
I20260929 06:33:17.951566 1791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:17.951840 1791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:17.953043 1791 hybrid_clock.cc:648] HybridClock initialized: now 1790663597953034 us; error 39 us; skew 500 ppm
Sep 29 06:33:17 dist-test-slave-sm0p krb5kdc[1778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:17.955489 1791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:17.955941 1791 webserver.cc:460] Webserver started at http://127.0.0.1:42517/ using document root <none> and password file <none>
I20260929 06:33:17.956175 1791 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:17.956239 1791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:17.956386 1791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:17.957414 1791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "93daf312094348b7a5afc54cf1b57567"
format_stamp: "Formatted at 2026-09-29 06:33:17 on dist-test-slave-sm0p"
server_key: "995cdd137ddfefe7b7bc1acf4d537e45"
server_key_iv: "ec536e230f29454d1f91b2223c4a5cdc"
server_key_version: "encryptionkey@0"
I20260929 06:33:17.957818 1791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "93daf312094348b7a5afc54cf1b57567"
format_stamp: "Formatted at 2026-09-29 06:33:17 on dist-test-slave-sm0p"
server_key: "995cdd137ddfefe7b7bc1acf4d537e45"
server_key_iv: "ec536e230f29454d1f91b2223c4a5cdc"
server_key_version: "encryptionkey@0"
I20260929 06:33:17.959252 1791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:17.959972 1806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:17.960139 1791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:17.960230 1791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "93daf312094348b7a5afc54cf1b57567"
format_stamp: "Formatted at 2026-09-29 06:33:17 on dist-test-slave-sm0p"
server_key: "995cdd137ddfefe7b7bc1acf4d537e45"
server_key_iv: "ec536e230f29454d1f91b2223c4a5cdc"
server_key_version: "encryptionkey@0"
I20260929 06:33:17.960309 1791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:17.967482 1791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:17.969712 1791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:17.969877 1791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:17.973796 1791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41395
I20260929 06:33:17.973834 1858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41395 every 8 connection(s)
I20260929 06:33:17.974263 1791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:17.975066 1859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:17.977195 1859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567: Bootstrap starting.
I20260929 06:33:17.977487 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1791
I20260929 06:33:17.977659 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:17.977833 1859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:17.977833 13374 external_mini_cluster.cc:1428] Setting key b376f73957f5c5cd9d9630e56779546f
I20260929 06:33:17.978114 1859 log.cc:824] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:17.978955 1859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567: No bootstrap required, opened a new log
I20260929 06:33:17.981244 1859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } }
I20260929 06:33:17.981405 1859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:17.981434 1859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93daf312094348b7a5afc54cf1b57567, State: Initialized, Role: FOLLOWER
I20260929 06:33:17.981556 1859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } }
I20260929 06:33:17.981639 1859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:17.981673 1859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:17.981722 1859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:17.982335 1859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } }
I20260929 06:33:17.982458 1859 leader_election.cc:302] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93daf312094348b7a5afc54cf1b57567; no voters:
I20260929 06:33:17.982609 1859 leader_election.cc:288] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:17.982667 1864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:17.982836 1864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [term 1 LEADER]: Becoming Leader. State: Replica: 93daf312094348b7a5afc54cf1b57567, State: Running, Role: LEADER
I20260929 06:33:17.982841 1859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:17.982932 1864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } }
I20260929 06:33:17.983261 1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93daf312094348b7a5afc54cf1b57567. Latest consensus state: current_term: 1 leader_uuid: "93daf312094348b7a5afc54cf1b57567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } } }
I20260929 06:33:17.983356 1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:17.983474 1865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93daf312094348b7a5afc54cf1b57567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93daf312094348b7a5afc54cf1b57567" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41395 } } }
I20260929 06:33:17.983531 1865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:17.984254 1871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:17.984711 1871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 06:33:17 dist-test-slave-sm0p krb5kdc[1778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 06:33:17.986496 1871 catalog_manager.cc:1347] Generated new cluster ID: 961e823947e1432186c30f86e6fda64e
I20260929 06:33:17.986548 1871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:17.989228 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:17.978381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:49313 (local address 127.0.0.1:41395)
0929 06:33:17.978543 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:17.978552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:17.978610 (+ 58us) server_negotiation.cc:407] Connection header received
0929 06:33:17.979194 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:17.979203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:17.979279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:17.979437 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:17.980028 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:17.981490 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:17.982356 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:17.983022 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:17.983134 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:17.985818 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:17.985821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:17.985822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:17.985834 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:17.987699 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:17.988095 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:17.988098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:17.988099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:17.988197 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:17.988433 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:17.988434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:17.988435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:17.988667 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:17.988865 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:17.988956 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 06:33:17.989055 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260929 06:33:17.990123 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:17.997102 1871 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:17.997767 1871 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:18.003227 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:17.996903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:50125 (local address 127.0.0.1:41395)
0929 06:33:17.996939 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:17.996941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:17.997202 (+ 261us) server_negotiation.cc:407] Connection header received
0929 06:33:17.997283 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:17.997284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:17.997295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:17.997358 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:17.997898 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:17.998438 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:17.999119 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:17.999376 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:17.999455 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:18.000647 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:18.000649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:18.000649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:18.000656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:18.002037 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:18.002433 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:18.002434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:18.002435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:18.002487 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:18.002713 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:18.002713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:18.002714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:18.002823 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:18.002937 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:18.003054 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 06:33:18.003116 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:33:18.013985 1871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93daf312094348b7a5afc54cf1b57567: Generated new TSK 0
I20260929 06:33:18.014212 1871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:18.016758 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1791
W20260929 06:33:18.021097 1776 connection.cc:535] client connection to 127.0.0.1:41395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41395): Connection reset by peer (error 104)
2026-09-29T06:33:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:18 dist-test-slave-sm0p krb5kdc[1890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:18 dist-test-slave-sm0p krb5kdc[1890](info): set up 2 sockets
Sep 29 06:33:18 dist-test-slave-sm0p krb5kdc[1890](info): commencing operation
krb5kdc: starting...
W20260929 06:33:19.078440 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:19 dist-test-slave-sm0p krb5kdc[1890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:19.193224 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:19.274719 1903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:19.274904 1903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:19.274927 1903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:19.276063 1903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:19.276117 1903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:19.276135 1903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:19.276191 1903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:19.276216 1903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:19.277736 1903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:19.277976 1903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:19.278255 1903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:19.283115 1909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:19.283162 1908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:19.283129 1911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:19.283468 1903 server_base.cc:1034] running on GCE node
I20260929 06:33:19.283675 1903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:19.283898 1903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:19.285036 1903 hybrid_clock.cc:648] HybridClock initialized: now 1790663599285025 us; error 28 us; skew 500 ppm
Sep 29 06:33:19 dist-test-slave-sm0p krb5kdc[1890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:19.287446 1903 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:19.287791 1903 webserver.cc:460] Webserver started at http://127.0.0.1:38287/ using document root <none> and password file <none>
I20260929 06:33:19.287993 1903 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:19.288029 1903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:19.288110 1903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:19.289029 1903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "b5f60032568843b49a0496d80a06138b"
format_stamp: "Formatted at 2026-09-29 06:33:19 on dist-test-slave-sm0p"
server_key: "7ba8062934bb6f596185fcaf9e3d3fbf"
server_key_iv: "da602d2597c8a84a1a55332ba2f679c9"
server_key_version: "encryptionkey@0"
I20260929 06:33:19.289304 1903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "b5f60032568843b49a0496d80a06138b"
format_stamp: "Formatted at 2026-09-29 06:33:19 on dist-test-slave-sm0p"
server_key: "7ba8062934bb6f596185fcaf9e3d3fbf"
server_key_iv: "da602d2597c8a84a1a55332ba2f679c9"
server_key_version: "encryptionkey@0"
I20260929 06:33:19.290552 1903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:19.291200 1918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:19.291409 1903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:19.291477 1903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "b5f60032568843b49a0496d80a06138b"
format_stamp: "Formatted at 2026-09-29 06:33:19 on dist-test-slave-sm0p"
server_key: "7ba8062934bb6f596185fcaf9e3d3fbf"
server_key_iv: "da602d2597c8a84a1a55332ba2f679c9"
server_key_version: "encryptionkey@0"
I20260929 06:33:19.291569 1903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:19.300158 1903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:19.301841 1903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:19.302002 1903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:19.305544 1903 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46191
I20260929 06:33:19.305594 1970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46191 every 8 connection(s)
I20260929 06:33:19.305904 1903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:19.306737 1971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:19.307600 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 1903
I20260929 06:33:19.307690 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:19.307904 13374 external_mini_cluster.cc:1428] Setting key 51822c031e9145734bafd685b4171595
I20260929 06:33:19.309445 1971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b: Bootstrap starting.
I20260929 06:33:19.310103 1971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:19.310412 1971 log.cc:824] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:19.311103 1971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b: No bootstrap required, opened a new log
I20260929 06:33:19.313417 1971 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260929 06:33:19.313580 1971 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:19.313637 1971 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f60032568843b49a0496d80a06138b, State: Initialized, Role: FOLLOWER
I20260929 06:33:19.313784 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:19.308430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:40641 (local address 127.0.0.1:46191)
0929 06:33:19.308605 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:19.308614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:19.308653 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:33:19.309166 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:19.309174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:19.309234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:19.309376 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:19.309567 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:19.310895 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:19.311665 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:19.312292 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:19.312440 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:19.313073 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:19.313078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:19.313079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:19.313267 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:19.313302 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:19.313316 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:19.313427 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:19.313507 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 06:33:19.313534 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260929 06:33:19.313769 1971 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260929 06:33:19.313844 1971 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:19.313876 1971 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:19.313912 1971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:19.314626 1971 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260929 06:33:19.314728 1971 leader_election.cc:302] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5f60032568843b49a0496d80a06138b; no voters:
I20260929 06:33:19.314819 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:19.314872 1971 leader_election.cc:288] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:19.314898 1976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:19.315032 1976 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [term 1 LEADER]: Becoming Leader. State: Replica: b5f60032568843b49a0496d80a06138b, State: Running, Role: LEADER
I20260929 06:33:19.315106 1971 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:19.315151 1976 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260929 06:33:19.315469 1977 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5f60032568843b49a0496d80a06138b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } } }
I20260929 06:33:19.315568 1977 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:19.315455 1976 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5f60032568843b49a0496d80a06138b. Latest consensus state: current_term: 1 leader_uuid: "b5f60032568843b49a0496d80a06138b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f60032568843b49a0496d80a06138b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } } }
I20260929 06:33:19.315771 1976 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:19.315891 1983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:19.316479 1983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:19.318145 1983 catalog_manager.cc:1347] Generated new cluster ID: fcfc082cb46d44108cfa39d87bcb70e4
I20260929 06:33:19.318223 1983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:19.320250 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:19.316475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:42813 (local address 127.0.0.1:46191)
0929 06:33:19.316535 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:19.316537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:19.316827 (+ 290us) server_negotiation.cc:407] Connection header received
0929 06:33:19.316929 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:19.316930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:19.316939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:19.317000 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:19.317182 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:19.317908 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:19.318961 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:19.319285 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:19.319373 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:19.319983 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:19.319984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:19.319985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:19.319988 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:19.320001 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:19.320006 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:19.320070 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:19.320166 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 06:33:19.320175 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 06:33:19.331624 1983 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:19.332123 1983 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:19.337787 1983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5f60032568843b49a0496d80a06138b: Generated new TSK 0
I20260929 06:33:19.337956 1983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:19.340530 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 1903
W20260929 06:33:19.344944 1888 connection.cc:535] client connection to 127.0.0.1:46191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46191): Connection reset by peer (error 104)
2026-09-29T06:33:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:19 dist-test-slave-sm0p krb5kdc[2002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:19 dist-test-slave-sm0p krb5kdc[2002](info): set up 2 sockets
Sep 29 06:33:19 dist-test-slave-sm0p krb5kdc[2002](info): commencing operation
krb5kdc: starting...
W20260929 06:33:20.350359 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:20 dist-test-slave-sm0p krb5kdc[2002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:20.462762 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:20.543742 2015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:20.543937 2015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:20.543962 2015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:20.545038 2015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:20.545078 2015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:20.545094 2015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:20.545107 2015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:20.545121 2015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:20.546540 2015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:20.546823 2015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:20.547058 2015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:20.551828 2021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:20.551838 2020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:20.551880 2023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:20.552259 2015 server_base.cc:1034] running on GCE node
I20260929 06:33:20.552491 2015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:20.552810 2015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:20.553979 2015 hybrid_clock.cc:648] HybridClock initialized: now 1790663600553962 us; error 37 us; skew 500 ppm
Sep 29 06:33:20 dist-test-slave-sm0p krb5kdc[2002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:20.556386 2015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:20.556844 2015 webserver.cc:460] Webserver started at http://127.0.0.1:40353/ using document root <none> and password file <none>
I20260929 06:33:20.557035 2015 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:20.557075 2015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:20.557161 2015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:20.558100 2015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "b9a16c048b9348a0a54a8640c8fec58a"
format_stamp: "Formatted at 2026-09-29 06:33:20 on dist-test-slave-sm0p"
server_key: "46565830a229a8f987ca2b7ea370f12b"
server_key_iv: "8e239912d9d7c6e3b1b50b6552b150b6"
server_key_version: "encryptionkey@0"
I20260929 06:33:20.558490 2015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "b9a16c048b9348a0a54a8640c8fec58a"
format_stamp: "Formatted at 2026-09-29 06:33:20 on dist-test-slave-sm0p"
server_key: "46565830a229a8f987ca2b7ea370f12b"
server_key_iv: "8e239912d9d7c6e3b1b50b6552b150b6"
server_key_version: "encryptionkey@0"
I20260929 06:33:20.559710 2015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:20.560353 2030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:20.560503 2015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:20.560570 2015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "b9a16c048b9348a0a54a8640c8fec58a"
format_stamp: "Formatted at 2026-09-29 06:33:20 on dist-test-slave-sm0p"
server_key: "46565830a229a8f987ca2b7ea370f12b"
server_key_iv: "8e239912d9d7c6e3b1b50b6552b150b6"
server_key_version: "encryptionkey@0"
I20260929 06:33:20.560617 2015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:20.563860 2015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:20.566380 2015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:20.566520 2015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:20.571540 2015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43853
I20260929 06:33:20.571566 2082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43853 every 8 connection(s)
I20260929 06:33:20.572021 2015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:20.573431 2083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:20.575682 2083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a: Bootstrap starting.
I20260929 06:33:20.576231 2083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:20.576521 2083 log.cc:824] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:20.577135 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2015
I20260929 06:33:20.577184 2083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a: No bootstrap required, opened a new log
I20260929 06:33:20.577261 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:20.577438 13374 external_mini_cluster.cc:1428] Setting key 6c7c721a880382d3ade00154895adb01
I20260929 06:33:20.579000 2083 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260929 06:33:20.579146 2083 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:20.579205 2083 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a16c048b9348a0a54a8640c8fec58a, State: Initialized, Role: FOLLOWER
I20260929 06:33:20.579355 2083 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260929 06:33:20.579479 2083 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:20.579512 2083 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:20.579559 2083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:20.580511 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:20.577912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:34877 (local address 127.0.0.1:43853)
0929 06:33:20.578089 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:20.578116 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:20.578200 (+ 84us) server_negotiation.cc:407] Connection header received
0929 06:33:20.578998 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:20.579006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:20.579036 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:20.579197 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:20.579863 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:20.579884 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:20.579884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:20.579891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:20.579933 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:20.579944 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:20.580171 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:20.580250 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 06:33:20.580291 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260929 06:33:20.580554 2083 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260929 06:33:20.580763 2083 leader_election.cc:302] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a16c048b9348a0a54a8640c8fec58a; no voters:
I20260929 06:33:20.580924 2083 leader_election.cc:288] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:20.581162 2083 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:20.581175 2088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:20.581415 2088 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [term 1 LEADER]: Becoming Leader. State: Replica: b9a16c048b9348a0a54a8640c8fec58a, State: Running, Role: LEADER
I20260929 06:33:20.581522 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:20.581539 2088 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260929 06:33:20.581844 2088 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a16c048b9348a0a54a8640c8fec58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260929 06:33:20.581909 2088 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:20.582037 2088 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a16c048b9348a0a54a8640c8fec58a. Latest consensus state: current_term: 1 leader_uuid: "b9a16c048b9348a0a54a8640c8fec58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a16c048b9348a0a54a8640c8fec58a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260929 06:33:20.582078 2088 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:20.582125 2097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:20.582645 2097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:20.584553 2097 catalog_manager.cc:1347] Generated new cluster ID: f26230cf889f4e88b66e4ca1b407a826
I20260929 06:33:20.584765 2097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:20.585091 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:20.583337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:49663 (local address 127.0.0.1:43853)
0929 06:33:20.583394 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:20.583396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:20.583938 (+ 542us) server_negotiation.cc:407] Connection header received
0929 06:33:20.584067 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:20.584067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:20.584072 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:20.584132 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:20.584753 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:20.584754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:20.584755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:20.584760 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:20.584777 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:20.584782 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:20.584854 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:20.585005 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 06:33:20.585020 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:33:20.622843 2097 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:20.623040 2097 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:20.627238 2097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a16c048b9348a0a54a8640c8fec58a: Generated new TSK 0
I20260929 06:33:20.627425 2097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:20.632975 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2015
2026-09-29T06:33:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:20 dist-test-slave-sm0p krb5kdc[2114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:20 dist-test-slave-sm0p krb5kdc[2114](info): set up 2 sockets
Sep 29 06:33:20 dist-test-slave-sm0p krb5kdc[2114](info): commencing operation
krb5kdc: starting...
W20260929 06:33:21.638161 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:21.748729 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:21.828063 2127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:21.828272 2127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:21.828308 2127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:21.829392 2127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:21.829455 2127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:21.829480 2127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:21.829509 2127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:21.829531 2127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:21.830981 2127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:21.831233 2127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:21.831441 2127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:21.836127 2133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:21.836134 2132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:21.836373 2127 server_base.cc:1034] running on GCE node
W20260929 06:33:21.836134 2135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:21.836596 2127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:21.836892 2127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:21.838091 2127 hybrid_clock.cc:648] HybridClock initialized: now 1790663601838069 us; error 30 us; skew 500 ppm
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:21.847438 2127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:21.847875 2127 webserver.cc:460] Webserver started at http://127.0.0.1:41113/ using document root <none> and password file <none>
I20260929 06:33:21.848104 2127 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:21.848182 2127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:21.848330 2127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:21.849296 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "a013e56667cc49bf947182441799fb11"
format_stamp: "Formatted at 2026-09-29 06:33:21 on dist-test-slave-sm0p"
server_key: "ce6180a70bf12f3d64c2effcbb1d94bf"
server_key_iv: "f28c39e2d0cb4f3032388b0a14ba7313"
server_key_version: "encryptionkey@0"
I20260929 06:33:21.849653 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "a013e56667cc49bf947182441799fb11"
format_stamp: "Formatted at 2026-09-29 06:33:21 on dist-test-slave-sm0p"
server_key: "ce6180a70bf12f3d64c2effcbb1d94bf"
server_key_iv: "f28c39e2d0cb4f3032388b0a14ba7313"
server_key_version: "encryptionkey@0"
I20260929 06:33:21.851110 2127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:21.851858 2142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:21.852030 2127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:21.852147 2127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "a013e56667cc49bf947182441799fb11"
format_stamp: "Formatted at 2026-09-29 06:33:21 on dist-test-slave-sm0p"
server_key: "ce6180a70bf12f3d64c2effcbb1d94bf"
server_key_iv: "f28c39e2d0cb4f3032388b0a14ba7313"
server_key_version: "encryptionkey@0"
I20260929 06:33:21.852226 2127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:21.861155 2127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:21.863444 2127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:21.863631 2127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:21.867579 2127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34363
I20260929 06:33:21.867621 2194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34363 every 8 connection(s)
I20260929 06:33:21.867940 2127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:21.868754 2195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:21.870898 2195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11: Bootstrap starting.
I20260929 06:33:21.871441 2195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:21.871697 2195 log.cc:824] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:21.872306 2195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11: No bootstrap required, opened a new log
I20260929 06:33:21.873234 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2127
I20260929 06:33:21.873366 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:21.873548 13374 external_mini_cluster.cc:1428] Setting key e44baa8d21db05174ee8c5d69137be95
I20260929 06:33:21.874179 2195 raft_consensus.cc:348] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260929 06:33:21.874368 2195 raft_consensus.cc:374] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:21.874405 2195 raft_consensus.cc:729] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a013e56667cc49bf947182441799fb11, State: Initialized, Role: FOLLOWER
I20260929 06:33:21.874524 2195 consensus_queue.cc:260] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260929 06:33:21.874599 2195 raft_consensus.cc:388] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:21.874634 2195 raft_consensus.cc:482] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:21.874672 2195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:21.875641 2195 raft_consensus.cc:504] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260929 06:33:21.875746 2195 leader_election.cc:302] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a013e56667cc49bf947182441799fb11; no voters:
I20260929 06:33:21.875890 2195 leader_election.cc:288] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:21.875959 2200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:21.876102 2195 sys_catalog.cc:564] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:21.876127 2200 raft_consensus.cc:686] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [term 1 LEADER]: Becoming Leader. State: Replica: a013e56667cc49bf947182441799fb11, State: Running, Role: LEADER
I20260929 06:33:21.876324 2200 consensus_queue.cc:237] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } }
I20260929 06:33:21.876744 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a013e56667cc49bf947182441799fb11. Latest consensus state: current_term: 1 leader_uuid: "a013e56667cc49bf947182441799fb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } } }
I20260929 06:33:21.876839 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:21.877143 2201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a013e56667cc49bf947182441799fb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a013e56667cc49bf947182441799fb11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34363 } } }
I20260929 06:33:21.877213 2201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:21.877665 2209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:21.878240 2209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:21.879870 2209 catalog_manager.cc:1347] Generated new cluster ID: 248d8a2087de4bd6ada6de54561e30ac
I20260929 06:33:21.879920 2209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 06:33:21.885150 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:21.874087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:42939 (local address 127.0.0.1:34363)
0929 06:33:21.874305 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:21.874314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:21.874354 (+ 40us) server_negotiation.cc:407] Connection header received
0929 06:33:21.874460 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:21.874467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:21.874521 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:21.874662 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:21.875303 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:21.876701 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:21.877795 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:21.878272 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:21.878397 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:21.881331 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:21.881334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:21.881335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:21.881347 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:21.883291 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:21.883720 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:21.883723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:21.883723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:21.883850 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:21.884084 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:21.884085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:21.884086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:21.884404 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:21.884570 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:21.884665 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 06:33:21.884782 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":129,"threads_started":1}
I20260929 06:33:21.886176 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:21.895998 2209 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:21.896431 2209 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:21.897183 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:21.887582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:40807 (local address 127.0.0.1:34363)
0929 06:33:21.887611 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:21.887612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:21.890289 (+ 2677us) server_negotiation.cc:407] Connection header received
0929 06:33:21.890318 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:21.890318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:21.890329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:21.890389 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:21.890951 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:21.891680 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:21.892604 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:21.892944 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:21.893038 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:21.894334 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:21.894335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:21.894335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:21.894342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:21.895915 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:21.896309 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:21.896310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:21.896310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:21.896359 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:21.896563 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:21.896564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:21.896564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:21.896710 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:21.896788 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:21.897025 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 06:33:21.897102 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:33:21.898545 2209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a013e56667cc49bf947182441799fb11: Generated new TSK 0
I20260929 06:33:21.898737 2209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:21.904743 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2127
W20260929 06:33:21.908521 2112 connection.cc:535] client connection to 127.0.0.1:34363 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34363): Connection reset by peer (error 104)
2026-09-29T06:33:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2226](info): set up 2 sockets
Sep 29 06:33:21 dist-test-slave-sm0p krb5kdc[2226](info): commencing operation
krb5kdc: starting...
W20260929 06:33:22.918318 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:22 dist-test-slave-sm0p krb5kdc[2226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:23.029294 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:23.108915 2239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:23.109107 2239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:23.109127 2239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:23.110186 2239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:23.110220 2239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:23.110234 2239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:23.110246 2239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:23.110257 2239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:23.111718 2239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:23.111893 2239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:23.112046 2239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:23.116748 2247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:23.116847 2239 server_base.cc:1034] running on GCE node
W20260929 06:33:23.116741 2245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:23.116741 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:23.117151 2239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:23.117383 2239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:23.118515 2239 hybrid_clock.cc:648] HybridClock initialized: now 1790663603118499 us; error 34 us; skew 500 ppm
Sep 29 06:33:23 dist-test-slave-sm0p krb5kdc[2226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663603, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:23.120755 2239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:23.121209 2239 webserver.cc:460] Webserver started at http://127.0.0.1:41287/ using document root <none> and password file <none>
I20260929 06:33:23.121443 2239 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:23.121519 2239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:23.121658 2239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:23.122699 2239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "e3284ebcc3404455aca68af6829a07e3"
format_stamp: "Formatted at 2026-09-29 06:33:23 on dist-test-slave-sm0p"
server_key: "ef08c36b611c1372e54b81c5e94b2ecc"
server_key_iv: "5d254988185e999c78bba5e8a5a8b2a1"
server_key_version: "encryptionkey@0"
I20260929 06:33:23.123075 2239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "e3284ebcc3404455aca68af6829a07e3"
format_stamp: "Formatted at 2026-09-29 06:33:23 on dist-test-slave-sm0p"
server_key: "ef08c36b611c1372e54b81c5e94b2ecc"
server_key_iv: "5d254988185e999c78bba5e8a5a8b2a1"
server_key_version: "encryptionkey@0"
I20260929 06:33:23.124411 2239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 06:33:23.125133 2254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:23.125425 2239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:23.125525 2239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "e3284ebcc3404455aca68af6829a07e3"
format_stamp: "Formatted at 2026-09-29 06:33:23 on dist-test-slave-sm0p"
server_key: "ef08c36b611c1372e54b81c5e94b2ecc"
server_key_iv: "5d254988185e999c78bba5e8a5a8b2a1"
server_key_version: "encryptionkey@0"
I20260929 06:33:23.125594 2239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:23.139773 2239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:23.141579 2239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:23.141747 2239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:23.146083 2239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36399
I20260929 06:33:23.146121 2306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36399 every 8 connection(s)
I20260929 06:33:23.146589 2239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:23.147411 2307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:23.149610 2307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3: Bootstrap starting.
I20260929 06:33:23.150210 2307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:23.150491 2307 log.cc:824] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:23.151166 2307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3: No bootstrap required, opened a new log
I20260929 06:33:23.153425 2307 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260929 06:33:23.153573 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2239
I20260929 06:33:23.153591 2307 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:23.153636 2307 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3284ebcc3404455aca68af6829a07e3, State: Initialized, Role: FOLLOWER
I20260929 06:33:23.153733 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:23.153820 2307 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260929 06:33:23.153929 2307 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:23.153972 13374 external_mini_cluster.cc:1428] Setting key c522e9414b363958cf61abefc36104e6
I20260929 06:33:23.153978 2307 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:23.154028 2307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:23.155226 2307 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260929 06:33:23.155344 2307 leader_election.cc:302] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3284ebcc3404455aca68af6829a07e3; no voters:
I20260929 06:33:23.155503 2307 leader_election.cc:288] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:23.155575 2312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:23.155741 2312 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [term 1 LEADER]: Becoming Leader. State: Replica: e3284ebcc3404455aca68af6829a07e3, State: Running, Role: LEADER
I20260929 06:33:23.155747 2307 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:23.155851 2312 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } }
I20260929 06:33:23.156212 2312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3284ebcc3404455aca68af6829a07e3. Latest consensus state: current_term: 1 leader_uuid: "e3284ebcc3404455aca68af6829a07e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } } }
I20260929 06:33:23.156440 2312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:23.156355 2313 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3284ebcc3404455aca68af6829a07e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3284ebcc3404455aca68af6829a07e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36399 } } }
I20260929 06:33:23.156810 2313 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:23.156910 2321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:23.157361 2321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:23.158932 2321 catalog_manager.cc:1347] Generated new cluster ID: 479850fd5d4540cd9b69d9c5d4fb18b2
I20260929 06:33:23.158993 2321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:23.163357 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:23.154516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:46641 (local address 127.0.0.1:36399)
0929 06:33:23.154764 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:23.154776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:23.154821 (+ 45us) server_negotiation.cc:407] Connection header received
0929 06:33:23.154955 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:23.154963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:23.155020 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:23.155179 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:23.155379 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:23.156817 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:23.157948 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:23.158545 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:23.158662 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:23.162715 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:23.162718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:23.162719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:23.162894 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:23.162927 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:23.162937 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:23.163031 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:23.163109 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 06:33:23.163153 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260929 06:33:23.164219 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:23.168305 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:23.165548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:54181 (local address 127.0.0.1:36399)
0929 06:33:23.165583 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:23.165585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:23.165685 (+ 100us) server_negotiation.cc:407] Connection header received
0929 06:33:23.165739 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:23.165740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:23.165750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:23.165797 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:23.165892 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:23.166426 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:23.167141 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:23.167354 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:23.167429 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:23.168056 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:23.168057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:23.168057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:23.168061 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:23.168074 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:23.168078 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:23.168140 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:23.168205 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 06:33:23.168217 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:33:23.184513 2321 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:23.185026 2321 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:23.187381 2321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3284ebcc3404455aca68af6829a07e3: Generated new TSK 0
I20260929 06:33:23.187551 2321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:23.189785 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2239
W20260929 06:33:23.193713 2224 connection.cc:535] client connection to 127.0.0.1:36399 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36399): Connection reset by peer (error 104)
2026-09-29T06:33:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:23 dist-test-slave-sm0p krb5kdc[2338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:23 dist-test-slave-sm0p krb5kdc[2338](info): set up 2 sockets
Sep 29 06:33:23 dist-test-slave-sm0p krb5kdc[2338](info): commencing operation
krb5kdc: starting...
W20260929 06:33:24.210350 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:24 dist-test-slave-sm0p krb5kdc[2338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 06:33:24.320811 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:24.397995 2351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:24.398247 2351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:24.398284 2351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:24.399421 2351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:24.399493 2351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:24.399514 2351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:24.399535 2351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:24.399564 2351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:24.401046 2351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38339
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:24.401317 2351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:24.401543 2351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:24.406416 2356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:24.406453 2359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:24.406487 2357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:24.406713 2351 server_base.cc:1034] running on GCE node
I20260929 06:33:24.406906 2351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:24.407155 2351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:24.408314 2351 hybrid_clock.cc:648] HybridClock initialized: now 1790663604408295 us; error 33 us; skew 500 ppm
Sep 29 06:33:24 dist-test-slave-sm0p krb5kdc[2338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663604, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:24.410754 2351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 06:33:24.411173 2351 webserver.cc:460] Webserver started at http://127.0.0.1:37713/ using document root <none> and password file <none>
I20260929 06:33:24.411391 2351 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:24.411433 2351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:24.411564 2351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:24.412524 2351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "ce3976ed0a62445aa0dbdd07b06596b2"
format_stamp: "Formatted at 2026-09-29 06:33:24 on dist-test-slave-sm0p"
server_key: "bbcd8e34d642f9c40221617581d1d72e"
server_key_iv: "3180e0f3dd56930af250af2e3bc12eb9"
server_key_version: "encryptionkey@0"
I20260929 06:33:24.412879 2351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "ce3976ed0a62445aa0dbdd07b06596b2"
format_stamp: "Formatted at 2026-09-29 06:33:24 on dist-test-slave-sm0p"
server_key: "bbcd8e34d642f9c40221617581d1d72e"
server_key_iv: "3180e0f3dd56930af250af2e3bc12eb9"
server_key_version: "encryptionkey@0"
I20260929 06:33:24.414227 2351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:33:24.414989 2366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:24.415210 2351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:24.415280 2351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "ce3976ed0a62445aa0dbdd07b06596b2"
format_stamp: "Formatted at 2026-09-29 06:33:24 on dist-test-slave-sm0p"
server_key: "bbcd8e34d642f9c40221617581d1d72e"
server_key_iv: "3180e0f3dd56930af250af2e3bc12eb9"
server_key_version: "encryptionkey@0"
I20260929 06:33:24.415362 2351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:24.418382 2351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:24.420197 2351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:24.420331 2351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:24.424018 2351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38339
I20260929 06:33:24.424048 2418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38339 every 8 connection(s)
I20260929 06:33:24.424343 2351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:24.424642 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2351
I20260929 06:33:24.424798 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:24.425002 13374 external_mini_cluster.cc:1428] Setting key 91e7a41efc68d3ee280b4b5fabfbfd04
I20260929 06:33:24.425277 2419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:24.427551 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:24.425474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:56675 (local address 127.0.0.1:38339)
0929 06:33:24.425656 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:24.425664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:24.425693 (+ 29us) server_negotiation.cc:407] Connection header received
0929 06:33:24.426375 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:24.426383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:24.426414 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:24.426550 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:24.427154 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:24.427158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:24.427158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:24.427163 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:24.427203 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:24.427215 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:24.427286 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:24.427337 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 06:33:24.427376 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260929 06:33:24.428370 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:24.428570 2419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2: Bootstrap starting.
I20260929 06:33:24.429096 2419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:24.429409 2419 log.cc:824] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:24.430202 2419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2: No bootstrap required, opened a new log
I20260929 06:33:24.430739 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:24.429741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:58241 (local address 127.0.0.1:38339)
0929 06:33:24.429795 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:24.429797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:24.429888 (+ 91us) server_negotiation.cc:407] Connection header received
0929 06:33:24.429929 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:24.429930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:24.429934 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:24.429994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:24.430493 (+ 499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:24.430494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:24.430494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:24.430498 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:24.430511 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:24.430515 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:24.430594 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:24.430669 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 06:33:24.430680 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 06:33:24.432128 2419 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } }
I20260929 06:33:24.432261 2419 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:24.432304 2419 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3976ed0a62445aa0dbdd07b06596b2, State: Initialized, Role: FOLLOWER
I20260929 06:33:24.432418 2419 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } }
I20260929 06:33:24.432519 2419 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:24.432559 2419 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:24.432605 2419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:24.433084 2419 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } }
I20260929 06:33:24.433189 2419 leader_election.cc:302] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce3976ed0a62445aa0dbdd07b06596b2; no voters:
I20260929 06:33:24.433331 2419 leader_election.cc:288] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:24.433382 2429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:24.433535 2429 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [term 1 LEADER]: Becoming Leader. State: Replica: ce3976ed0a62445aa0dbdd07b06596b2, State: Running, Role: LEADER
I20260929 06:33:24.433559 2419 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:24.433696 2429 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } }
I20260929 06:33:24.434106 2430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } } }
I20260929 06:33:24.434160 2431 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce3976ed0a62445aa0dbdd07b06596b2. Latest consensus state: current_term: 1 leader_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3976ed0a62445aa0dbdd07b06596b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38339 } } }
I20260929 06:33:24.434284 2430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:24.434295 2431 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:24.434728 2438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:24.435115 2438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:24.436475 2438 catalog_manager.cc:1347] Generated new cluster ID: ef7524e8d127480baa31fb5af21ead06
I20260929 06:33:24.436523 2438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:24.446938 2438 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:24.447098 2438 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:24.454072 2438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce3976ed0a62445aa0dbdd07b06596b2: Generated new TSK 0
I20260929 06:33:24.454264 2438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:24.458854 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2351
2026-09-29T06:33:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:24 dist-test-slave-sm0p krb5kdc[2450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:24 dist-test-slave-sm0p krb5kdc[2450](info): set up 2 sockets
Sep 29 06:33:24 dist-test-slave-sm0p krb5kdc[2450](info): commencing operation
krb5kdc: starting...
W20260929 06:33:25.466194 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:25.578395 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43773
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:43773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:25.654114 2463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:25.654368 2463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:25.654410 2463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:25.655503 2463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:25.655566 2463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:25.655588 2463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:25.655607 2463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:25.655632 2463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:25.656908 2463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:43773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:25.657150 2463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:25.657346 2463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:25.662106 2468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:25.662115 2471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:25.662122 2469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:25.662181 2463 server_base.cc:1034] running on GCE node
I20260929 06:33:25.662580 2463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:25.662879 2463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:25.664037 2463 hybrid_clock.cc:648] HybridClock initialized: now 1790663605664017 us; error 37 us; skew 500 ppm
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663605, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:25.666430 2463 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:25.666841 2463 webserver.cc:460] Webserver started at http://127.13.15.190:46517/ using document root <none> and password file <none>
I20260929 06:33:25.667008 2463 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:25.667045 2463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:25.667126 2463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:25.668041 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "e2bbf5599423451c82dfcec41224e7fa"
format_stamp: "Formatted at 2026-09-29 06:33:25 on dist-test-slave-sm0p"
server_key: "5fbf24e31322bbd8b1fecdb1e83e14b9"
server_key_iv: "0dc29220d9dec8d4593942839b8ac19f"
server_key_version: "encryptionkey@0"
I20260929 06:33:25.668336 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "e2bbf5599423451c82dfcec41224e7fa"
format_stamp: "Formatted at 2026-09-29 06:33:25 on dist-test-slave-sm0p"
server_key: "5fbf24e31322bbd8b1fecdb1e83e14b9"
server_key_iv: "0dc29220d9dec8d4593942839b8ac19f"
server_key_version: "encryptionkey@0"
I20260929 06:33:25.669590 2463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:33:25.670293 2478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:25.670449 2463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:25.670527 2463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "e2bbf5599423451c82dfcec41224e7fa"
format_stamp: "Formatted at 2026-09-29 06:33:25 on dist-test-slave-sm0p"
server_key: "5fbf24e31322bbd8b1fecdb1e83e14b9"
server_key_iv: "0dc29220d9dec8d4593942839b8ac19f"
server_key_version: "encryptionkey@0"
I20260929 06:33:25.670576 2463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:25.693552 2463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:25.695422 2463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:25.695528 2463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:25.699254 2463 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:43773
I20260929 06:33:25.699325 2530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:43773 every 8 connection(s)
I20260929 06:33:25.699663 2463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:25.700492 2531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:25.702420 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2463
I20260929 06:33:25.702515 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:25.702755 13374 external_mini_cluster.cc:1428] Setting key 75950ec9390891f29bd4e79bc2143e93
I20260929 06:33:25.702823 2531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: Bootstrap starting.
I20260929 06:33:25.703531 2531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:25.703817 2531 log.cc:824] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:25.704504 2531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: No bootstrap required, opened a new log
I20260929 06:33:25.706377 2531 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } }
I20260929 06:33:25.706530 2531 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:25.706579 2531 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bbf5599423451c82dfcec41224e7fa, State: Initialized, Role: FOLLOWER
I20260929 06:33:25.706741 2531 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } }
I20260929 06:33:25.706832 2531 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:25.706881 2531 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:25.706930 2531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:25.707439 2531 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } }
I20260929 06:33:25.707556 2531 leader_election.cc:302] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2bbf5599423451c82dfcec41224e7fa; no voters:
I20260929 06:33:25.707705 2531 leader_election.cc:288] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:25.707727 2536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:25.707875 2536 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [term 1 LEADER]: Becoming Leader. State: Replica: e2bbf5599423451c82dfcec41224e7fa, State: Running, Role: LEADER
I20260929 06:33:25.707964 2536 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } }
I20260929 06:33:25.708279 2531 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:25.708280 2538 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2bbf5599423451c82dfcec41224e7fa. Latest consensus state: current_term: 1 leader_uuid: "e2bbf5599423451c82dfcec41224e7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } } }
I20260929 06:33:25.708416 2538 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:25.709573 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2bbf5599423451c82dfcec41224e7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bbf5599423451c82dfcec41224e7fa" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43773 } } }
I20260929 06:33:25.709666 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa [sys.catalog]: This master's current role is: LEADER
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
W20260929 06:33:25.710124 2552 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:33:25.710436 2552 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:25.710696 2540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:25.710927 2540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:25.712764 2540 catalog_manager.cc:1347] Generated new cluster ID: 33e224d2c4734d6f827b06a2ce8a0e25
I20260929 06:33:25.712821 2540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:25.714651 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:25.703297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57369 (local address 127.13.15.190:43773)
0929 06:33:25.703527 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:25.703537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:25.703570 (+ 33us) server_negotiation.cc:407] Connection header received
0929 06:33:25.704169 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:25.704176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:25.704224 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:25.704406 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:25.705143 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:25.706026 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:25.706965 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:25.707386 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:25.707546 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:25.710691 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:25.710695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:25.710696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:25.710709 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:25.712807 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:25.713282 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:25.713286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:25.713286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:25.713397 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:25.713673 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:25.713674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:25.713674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:25.714012 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:25.714201 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:25.714333 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 06:33:25.714441 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":122,"threads_started":1}
I20260929 06:33:25.715694 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:25.724535 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:25.717305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56129 (local address 127.13.15.190:43773)
0929 06:33:25.717339 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:25.717341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:25.717350 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:25.717395 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:25.717396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:25.717405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:25.717462 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:25.718069 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:25.718824 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:25.719810 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:25.720102 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:25.720166 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:25.721563 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:25.721565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:25.721565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:25.721572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:25.723173 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:25.723623 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:25.723624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:25.723624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:25.723683 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:25.723928 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:25.723928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:25.723928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:25.724046 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:25.724145 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:25.724287 (+ 142us) server_negotiation.cc:299] Negotiation successful
0929 06:33:25.724348 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:33:25.726907 2540 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:25.727403 2540 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:25.732046 2540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2bbf5599423451c82dfcec41224e7fa: Generated new TSK 0
I20260929 06:33:25.732327 2540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:25.738574 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2463
W20260929 06:33:25.742836 2448 connection.cc:535] client connection to 127.13.15.190:43773 recv error: Network error: failed to read from TLS socket (remote: 127.13.15.190:43773): Connection reset by peer (error 104)
2026-09-29T06:33:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2562](info): set up 2 sockets
Sep 29 06:33:25 dist-test-slave-sm0p krb5kdc[2562](info): commencing operation
krb5kdc: starting...
W20260929 06:33:26.762430 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:26 dist-test-slave-sm0p krb5kdc[2562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:26.878473 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36041
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:36041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:26.957486 2575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:26.957667 2575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:26.957690 2575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:26.958801 2575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:26.958840 2575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:26.958861 2575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:26.958874 2575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:26.958886 2575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:26.960309 2575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:36041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:26.960551 2575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:26.960775 2575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:26.965530 2580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:26.965521 2583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:26.965531 2581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:26.965782 2575 server_base.cc:1034] running on GCE node
I20260929 06:33:26.966023 2575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:26.966315 2575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:26.967482 2575 hybrid_clock.cc:648] HybridClock initialized: now 1790663606967459 us; error 38 us; skew 500 ppm
Sep 29 06:33:26 dist-test-slave-sm0p krb5kdc[2562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663606, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:26.973462 2575 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:26.973917 2575 webserver.cc:460] Webserver started at http://127.13.15.190:34379/ using document root <none> and password file <none>
I20260929 06:33:26.974201 2575 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:26.974267 2575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:26.974421 2575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:26.975461 2575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "4511d24e12424d41a825dbf4f2fd15a3"
format_stamp: "Formatted at 2026-09-29 06:33:26 on dist-test-slave-sm0p"
server_key: "7ad016fd1386f88d623f2ebdc2665b0b"
server_key_iv: "e280679d8f01395b5257470343ebd26b"
server_key_version: "encryptionkey@0"
I20260929 06:33:26.975809 2575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "4511d24e12424d41a825dbf4f2fd15a3"
format_stamp: "Formatted at 2026-09-29 06:33:26 on dist-test-slave-sm0p"
server_key: "7ad016fd1386f88d623f2ebdc2665b0b"
server_key_iv: "e280679d8f01395b5257470343ebd26b"
server_key_version: "encryptionkey@0"
I20260929 06:33:26.977080 2575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:26.977809 2590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:26.977991 2575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 06:33:26.978086 2575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "4511d24e12424d41a825dbf4f2fd15a3"
format_stamp: "Formatted at 2026-09-29 06:33:26 on dist-test-slave-sm0p"
server_key: "7ad016fd1386f88d623f2ebdc2665b0b"
server_key_iv: "e280679d8f01395b5257470343ebd26b"
server_key_version: "encryptionkey@0"
I20260929 06:33:26.978194 2575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:26.991438 2575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:26.993182 2575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:26.993335 2575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:26.997151 2575 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:36041
I20260929 06:33:26.997184 2642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:36041 every 8 connection(s)
I20260929 06:33:26.997498 2575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:26.998350 2643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:27.000407 2643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3: Bootstrap starting.
I20260929 06:33:27.000885 2643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:27.001111 2643 log.cc:824] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:27.001698 2643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3: No bootstrap required, opened a new log
I20260929 06:33:27.002624 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2575
I20260929 06:33:27.002722 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:27.002892 13374 external_mini_cluster.cc:1428] Setting key 50fa3cd739acd2a748150497e84c7121
I20260929 06:33:27.003577 2643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } }
I20260929 06:33:27.003748 2643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:27.003782 2643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4511d24e12424d41a825dbf4f2fd15a3, State: Initialized, Role: FOLLOWER
I20260929 06:33:27.003901 2643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } }
I20260929 06:33:27.003983 2643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:27.004016 2643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:27.004105 2643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:27.004833 2643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } }
I20260929 06:33:27.004937 2643 leader_election.cc:302] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4511d24e12424d41a825dbf4f2fd15a3; no voters:
I20260929 06:33:27.005079 2643 leader_election.cc:288] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:27.005139 2648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:27.005275 2648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [term 1 LEADER]: Becoming Leader. State: Replica: 4511d24e12424d41a825dbf4f2fd15a3, State: Running, Role: LEADER
I20260929 06:33:27.005317 2643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:27.005420 2648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } }
I20260929 06:33:27.005993 2650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4511d24e12424d41a825dbf4f2fd15a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } } }
I20260929 06:33:27.006100 2650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:27.005985 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4511d24e12424d41a825dbf4f2fd15a3. Latest consensus state: current_term: 1 leader_uuid: "4511d24e12424d41a825dbf4f2fd15a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4511d24e12424d41a825dbf4f2fd15a3" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36041 } } }
I20260929 06:33:27.006300 2657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:27.006374 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:27.006898 2657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:27.008795 2657 catalog_manager.cc:1347] Generated new cluster ID: cf7c9138edf14a358916a0e332d1084a
I20260929 06:33:27.008841 2657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:27.009259 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:27.003401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41307 (local address 127.13.15.190:36041)
0929 06:33:27.003621 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:27.003630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:27.003670 (+ 40us) server_negotiation.cc:407] Connection header received
0929 06:33:27.003774 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:27.003781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:27.003836 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:27.003981 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:27.004218 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:27.005864 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:27.007130 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:27.007824 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:27.007917 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:27.008616 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:27.008619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:27.008619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:27.008814 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:27.008850 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:27.008862 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:27.008942 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:27.009054 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 06:33:27.009080 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260929 06:33:27.010177 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:27.015093 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:27.011588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35867 (local address 127.13.15.190:36041)
0929 06:33:27.011618 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:27.011619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:27.011644 (+ 25us) server_negotiation.cc:407] Connection header received
0929 06:33:27.011712 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:27.011713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:27.011721 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:27.011767 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:27.011883 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:27.012527 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:27.013507 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:27.014090 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:27.014191 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:27.014847 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:27.014848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:27.014849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:27.014853 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:27.014868 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:27.014873 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:27.014929 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:27.015013 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 06:33:27.015025 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 06:33:27.018061 2657 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:27.018584 2657 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:27.021402 2657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4511d24e12424d41a825dbf4f2fd15a3: Generated new TSK 0
I20260929 06:33:27.021565 2657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:27.029409 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2575
W20260929 06:33:27.033061 2560 connection.cc:535] client connection to 127.13.15.190:36041 recv error: Network error: failed to read from TLS socket (remote: 127.13.15.190:36041): Connection reset by peer (error 104)
2026-09-29T06:33:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:27 dist-test-slave-sm0p krb5kdc[2674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:27 dist-test-slave-sm0p krb5kdc[2674](info): set up 2 sockets
Sep 29 06:33:27 dist-test-slave-sm0p krb5kdc[2674](info): commencing operation
krb5kdc: starting...
W20260929 06:33:28.030077 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:28 dist-test-slave-sm0p krb5kdc[2674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:28.142871 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44571
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:28.219753 2687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:28.219947 2687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:28.219969 2687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:28.220996 2687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:28.221030 2687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:28.221043 2687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:28.221055 2687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:28.221067 2687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:28.222401 2687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:44571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:28.222565 2687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:28.222723 2687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:28.227372 2692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:28.227353 2695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:28.227380 2693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:28.227506 2687 server_base.cc:1034] running on GCE node
I20260929 06:33:28.227828 2687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:28.228107 2687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:28.229275 2687 hybrid_clock.cc:648] HybridClock initialized: now 1790663608229264 us; error 42 us; skew 500 ppm
Sep 29 06:33:28 dist-test-slave-sm0p krb5kdc[2674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:28.231523 2687 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:28.231914 2687 webserver.cc:460] Webserver started at http://127.13.15.190:44997/ using document root <none> and password file <none>
I20260929 06:33:28.232120 2687 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:28.232200 2687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:28.232342 2687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:28.233289 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "5d5d42661b114a52b708e1c0775dcdcc"
format_stamp: "Formatted at 2026-09-29 06:33:28 on dist-test-slave-sm0p"
server_key: "54ee53ac476a99d38aef60e7f7d1a2a7"
server_key_iv: "be5e8c3114fa4beb2d5d57fe71ebb425"
server_key_version: "encryptionkey@0"
I20260929 06:33:28.233625 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "5d5d42661b114a52b708e1c0775dcdcc"
format_stamp: "Formatted at 2026-09-29 06:33:28 on dist-test-slave-sm0p"
server_key: "54ee53ac476a99d38aef60e7f7d1a2a7"
server_key_iv: "be5e8c3114fa4beb2d5d57fe71ebb425"
server_key_version: "encryptionkey@0"
I20260929 06:33:28.235023 2687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:28.235723 2702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:28.235884 2687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:28.235970 2687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "5d5d42661b114a52b708e1c0775dcdcc"
format_stamp: "Formatted at 2026-09-29 06:33:28 on dist-test-slave-sm0p"
server_key: "54ee53ac476a99d38aef60e7f7d1a2a7"
server_key_iv: "be5e8c3114fa4beb2d5d57fe71ebb425"
server_key_version: "encryptionkey@0"
I20260929 06:33:28.236040 2687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:28.239198 2687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:28.240846 2687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:28.241022 2687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:28.244678 2687 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:44571
I20260929 06:33:28.244726 2754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:44571 every 8 connection(s)
I20260929 06:33:28.245028 2687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:28.245954 2755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:28.247418 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2687
I20260929 06:33:28.247551 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:28.247730 13374 external_mini_cluster.cc:1428] Setting key 7ec479866d40b3f9a0c54acdddfb888d
I20260929 06:33:28.248275 2755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc: Bootstrap starting.
I20260929 06:33:28.248926 2755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:28.249254 2755 log.cc:824] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:28.249996 2755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc: No bootstrap required, opened a new log
I20260929 06:33:28.250180 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:28.248183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45785 (local address 127.13.15.190:44571)
0929 06:33:28.248367 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:28.248376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:28.248409 (+ 33us) server_negotiation.cc:407] Connection header received
0929 06:33:28.248962 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:28.248971 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:28.249003 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:28.249154 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:28.249703 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:28.249706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:28.249706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:28.249710 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:28.249748 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:28.249765 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:28.249828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:28.249920 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 06:33:28.249969 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260929 06:33:28.251039 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:28.252462 2755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } }
I20260929 06:33:28.252633 2755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:28.252715 2755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5d42661b114a52b708e1c0775dcdcc, State: Initialized, Role: FOLLOWER
I20260929 06:33:28.252871 2755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } }
I20260929 06:33:28.252978 2755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:28.253026 2755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:28.253077 2755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:28.253680 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:28.252623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45455 (local address 127.13.15.190:44571)
0929 06:33:28.252686 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:28.252689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:28.252797 (+ 108us) server_negotiation.cc:407] Connection header received
0929 06:33:28.252875 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:28.252875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:28.252880 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:28.252934 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:28.253452 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:28.253453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:28.253454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:28.253458 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:28.253468 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:28.253472 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:28.253525 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:28.253596 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 06:33:28.253610 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 06:33:28.253684 2755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } }
I20260929 06:33:28.253835 2755 leader_election.cc:302] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5d42661b114a52b708e1c0775dcdcc; no voters:
I20260929 06:33:28.253996 2755 leader_election.cc:288] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:28.254026 2765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:28.254190 2765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [term 1 LEADER]: Becoming Leader. State: Replica: 5d5d42661b114a52b708e1c0775dcdcc, State: Running, Role: LEADER
I20260929 06:33:28.254251 2755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:28.254288 2765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } }
I20260929 06:33:28.254602 2765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5d42661b114a52b708e1c0775dcdcc. Latest consensus state: current_term: 1 leader_uuid: "5d5d42661b114a52b708e1c0775dcdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } } }
I20260929 06:33:28.254616 2766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d5d42661b114a52b708e1c0775dcdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5d42661b114a52b708e1c0775dcdcc" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 44571 } } }
I20260929 06:33:28.254746 2765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:28.254801 2766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:28.255167 2774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:28.255671 2774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:28.257081 2774 catalog_manager.cc:1347] Generated new cluster ID: dffa2159330c4049b7d0f87a442d2441
I20260929 06:33:28.257126 2774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:28.266660 2774 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:28.266803 2774 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:28.270805 2774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d5d42661b114a52b708e1c0775dcdcc: Generated new TSK 0
I20260929 06:33:28.270954 2774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:28.274528 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2687
2026-09-29T06:33:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:28 dist-test-slave-sm0p krb5kdc[2786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:28 dist-test-slave-sm0p krb5kdc[2786](info): set up 2 sockets
Sep 29 06:33:28 dist-test-slave-sm0p krb5kdc[2786](info): commencing operation
krb5kdc: starting...
W20260929 06:33:29.269970 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:29.381263 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43003
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:34067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:43003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:29.460578 2799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:29.460857 2799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:29.460896 2799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:29.461992 2799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:29.462071 2799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:29.462100 2799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:29.462131 2799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:29.462152 2799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:29.463914 2799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:34067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:43003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:29.464185 2799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:29.464416 2799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:29.469122 2804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:29.469166 2807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:29.469261 2805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:29.469458 2799 server_base.cc:1034] running on GCE node
I20260929 06:33:29.469729 2799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:29.469962 2799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:29.471153 2799 hybrid_clock.cc:648] HybridClock initialized: now 1790663609471123 us; error 38 us; skew 500 ppm
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:29.473380 2799 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:29.473831 2799 webserver.cc:460] Webserver started at http://127.13.15.190:40647/ using document root <none> and password file <none>
I20260929 06:33:29.474068 2799 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:29.474134 2799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:29.474349 2799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:29.475344 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "42e22ff984ca4541bfe41d73419cdde9"
format_stamp: "Formatted at 2026-09-29 06:33:29 on dist-test-slave-sm0p"
server_key: "cdff60bdf024da06684840b6537df2b5"
server_key_iv: "e572da456f813611ddc08741f11a7e16"
server_key_version: "encryptionkey@0"
I20260929 06:33:29.475703 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "42e22ff984ca4541bfe41d73419cdde9"
format_stamp: "Formatted at 2026-09-29 06:33:29 on dist-test-slave-sm0p"
server_key: "cdff60bdf024da06684840b6537df2b5"
server_key_iv: "e572da456f813611ddc08741f11a7e16"
server_key_version: "encryptionkey@0"
I20260929 06:33:29.476964 2799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:29.477619 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:29.477787 2799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:29.477888 2799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "42e22ff984ca4541bfe41d73419cdde9"
format_stamp: "Formatted at 2026-09-29 06:33:29 on dist-test-slave-sm0p"
server_key: "cdff60bdf024da06684840b6537df2b5"
server_key_iv: "e572da456f813611ddc08741f11a7e16"
server_key_version: "encryptionkey@0"
I20260929 06:33:29.477960 2799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:29.486696 2799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:29.488317 2799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:29.488476 2799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:29.492257 2799 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:43003
I20260929 06:33:29.492283 2866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:43003 every 8 connection(s)
I20260929 06:33:29.492628 2799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:29.493404 2867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:29.495203 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2799
I20260929 06:33:29.495325 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:29.495487 13374 external_mini_cluster.cc:1428] Setting key e7d54a97da0ef02c42626a9c7957d89f
I20260929 06:33:29.496086 2867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9: Bootstrap starting.
I20260929 06:33:29.496598 2867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:29.496861 2867 log.cc:824] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:29.497555 2867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9: No bootstrap required, opened a new log
I20260929 06:33:29.499321 2867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } }
I20260929 06:33:29.499467 2867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:29.499512 2867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e22ff984ca4541bfe41d73419cdde9, State: Initialized, Role: FOLLOWER
I20260929 06:33:29.499658 2867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } }
I20260929 06:33:29.499750 2867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:29.499806 2867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:29.499855 2867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:29.500341 2867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } }
I20260929 06:33:29.500452 2867 leader_election.cc:302] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42e22ff984ca4541bfe41d73419cdde9; no voters:
I20260929 06:33:29.500619 2867 leader_election.cc:288] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:29.500653 2872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:29.500759 2872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [term 1 LEADER]: Becoming Leader. State: Replica: 42e22ff984ca4541bfe41d73419cdde9, State: Running, Role: LEADER
I20260929 06:33:29.500859 2867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:29.500880 2872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } }
I20260929 06:33:29.501186 2872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42e22ff984ca4541bfe41d73419cdde9. Latest consensus state: current_term: 1 leader_uuid: "42e22ff984ca4541bfe41d73419cdde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } } }
I20260929 06:33:29.501199 2873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42e22ff984ca4541bfe41d73419cdde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42e22ff984ca4541bfe41d73419cdde9" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 43003 } } }
I20260929 06:33:29.501300 2872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:29.501533 2873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9 [sys.catalog]: This master's current role is: LEADER
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:29.501724 2879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:29.502518 2879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:29.504184 2879 catalog_manager.cc:1347] Generated new cluster ID: d0f6c112fcd94dbe95bd9f0d852aedd9
I20260929 06:33:29.504233 2879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:29.506346 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:29.495928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47875 (local address 127.13.15.190:43003)
0929 06:33:29.496076 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:29.496085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:29.496119 (+ 34us) server_negotiation.cc:407] Connection header received
0929 06:33:29.496517 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:29.496522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:29.496563 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:29.496715 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:29.497440 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:29.498411 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:29.499180 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:29.499667 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:29.499777 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:29.502503 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:29.502507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:29.502508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:29.502521 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:29.504523 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:29.504944 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:29.504947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:29.504947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:29.505046 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:29.505260 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:29.505261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:29.505262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:29.505554 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:29.505665 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:29.505961 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 06:33:29.506101 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260929 06:33:29.507493 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:29.516150 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:29.508869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48881 (local address 127.13.15.190:43003)
0929 06:33:29.508901 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:29.508902 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:29.508995 (+ 93us) server_negotiation.cc:407] Connection header received
0929 06:33:29.509038 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:29.509039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:29.509048 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:29.509102 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:29.509644 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:29.510481 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:29.511412 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:29.511740 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:29.511810 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:29.513294 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:29.513295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:29.513296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:29.513303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:29.514971 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:29.515348 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:29.515349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:29.515349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:29.515402 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:29.515595 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:29.515595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:29.515596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:29.515708 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:29.515785 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:29.515987 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 06:33:29.516059 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":16}
I20260929 06:33:29.531718 2879 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:29.532477 2879 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:29.538582 2879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42e22ff984ca4541bfe41d73419cdde9: Generated new TSK 0
I20260929 06:33:29.538757 2879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:29.542609 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2799
2026-09-29T06:33:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1268 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2898](info): set up 2 sockets
Sep 29 06:33:29 dist-test-slave-sm0p krb5kdc[2898](info): commencing operation
krb5kdc: starting...
W20260929 06:33:30.554333 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:30 dist-test-slave-sm0p krb5kdc[2898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:30.664994 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36441
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:36441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:30.741699 2911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:30.741940 2911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:30.741974 2911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:30.743191 2911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:30.743261 2911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:30.743283 2911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:30.743305 2911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:30.743330 2911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:30.744771 2911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:36441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:36441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:30.745038 2911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:30.745261 2911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:30.749959 2919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:30.750012 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:30.750088 2911 server_base.cc:1034] running on GCE node
W20260929 06:33:30.750191 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:30.750444 2911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:30.750735 2911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:30.751904 2911 hybrid_clock.cc:648] HybridClock initialized: now 1790663610751898 us; error 35 us; skew 500 ppm
Sep 29 06:33:30 dist-test-slave-sm0p krb5kdc[2898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663610, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:30.754230 2911 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:30.754645 2911 webserver.cc:460] Webserver started at http://127.13.15.190:43801/ using document root <none> and password file <none>
I20260929 06:33:30.754863 2911 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:30.754937 2911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:30.755092 2911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:30.756098 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "c7fc970a0e554cf98b2a14296d12f604"
format_stamp: "Formatted at 2026-09-29 06:33:30 on dist-test-slave-sm0p"
server_key: "6a388cc7818f59c8e62b2a0b4001ab47"
server_key_iv: "e38a9c46333eb06a64e522ec5d9abe53"
server_key_version: "encryptionkey@0"
I20260929 06:33:30.756453 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "c7fc970a0e554cf98b2a14296d12f604"
format_stamp: "Formatted at 2026-09-29 06:33:30 on dist-test-slave-sm0p"
server_key: "6a388cc7818f59c8e62b2a0b4001ab47"
server_key_iv: "e38a9c46333eb06a64e522ec5d9abe53"
server_key_version: "encryptionkey@0"
I20260929 06:33:30.757788 2911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:30.758594 2926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:30.758761 2911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:30.758859 2911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "c7fc970a0e554cf98b2a14296d12f604"
format_stamp: "Formatted at 2026-09-29 06:33:30 on dist-test-slave-sm0p"
server_key: "6a388cc7818f59c8e62b2a0b4001ab47"
server_key_iv: "e38a9c46333eb06a64e522ec5d9abe53"
server_key_version: "encryptionkey@0"
I20260929 06:33:30.758945 2911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:30.775029 2911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:30.776940 2911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:30.777102 2911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:30.781224 2911 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:36441
I20260929 06:33:30.781261 2978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:36441 every 8 connection(s)
I20260929 06:33:30.781611 2911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:30.782531 2979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:30.784687 2979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604: Bootstrap starting.
I20260929 06:33:30.785226 2979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:30.785449 2979 log.cc:824] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:30.786077 2979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604: No bootstrap required, opened a new log
I20260929 06:33:30.787647 2979 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } }
I20260929 06:33:30.787781 2979 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:30.787804 2979 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7fc970a0e554cf98b2a14296d12f604, State: Initialized, Role: FOLLOWER
I20260929 06:33:30.787900 2979 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } }
I20260929 06:33:30.787947 2979 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:30.787967 2979 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:30.787992 2979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:30.788738 2979 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } }
I20260929 06:33:30.788818 2979 leader_election.cc:302] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7fc970a0e554cf98b2a14296d12f604; no voters:
I20260929 06:33:30.788970 2979 leader_election.cc:288] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:30.789003 2982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:30.789117 2982 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [term 1 LEADER]: Becoming Leader. State: Replica: c7fc970a0e554cf98b2a14296d12f604, State: Running, Role: LEADER
I20260929 06:33:30.789237 2979 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:30.789261 2982 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } }
I20260929 06:33:30.789726 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 2911
I20260929 06:33:30.789844 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:30.789762 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7fc970a0e554cf98b2a14296d12f604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } } }
I20260929 06:33:30.789758 2984 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7fc970a0e554cf98b2a14296d12f604. Latest consensus state: current_term: 1 leader_uuid: "c7fc970a0e554cf98b2a14296d12f604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7fc970a0e554cf98b2a14296d12f604" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 36441 } } }
I20260929 06:33:30.789898 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:30.789907 2984 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:30.790063 13374 external_mini_cluster.cc:1428] Setting key 4012a6edaba573e2cc0100216a2b816d
I20260929 06:33:30.790259 2991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:30.790830 2991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:30.793849 2991 catalog_manager.cc:1347] Generated new cluster ID: 84a5e6b7066447cb951144026a4d064e
I20260929 06:33:30.793907 2991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:30.796358 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:30.790777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43633 (local address 127.13.15.190:36441)
0929 06:33:30.791161 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:30.791171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:30.791210 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:33:30.791375 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:30.791381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:30.791459 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:30.791596 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:30.791870 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:30.793373 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:30.794438 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:30.794875 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:30.795007 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:30.795709 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:30.795713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:30.795713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:30.795889 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:30.795919 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:30.795930 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:30.796002 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:30.796087 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 06:33:30.796152 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":77,"threads_started":1}
I20260929 06:33:30.797252 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:30.805176 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:30.802343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55889 (local address 127.13.15.190:36441)
0929 06:33:30.802389 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:30.802391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:30.802525 (+ 134us) server_negotiation.cc:407] Connection header received
0929 06:33:30.802597 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:30.802598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:30.802609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:30.802661 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:30.802780 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:30.803257 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:30.803983 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:30.804201 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:30.804278 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:30.804948 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:30.804949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:30.804950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:30.804955 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:30.804969 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:30.804973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:30.805021 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:30.805078 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:33:30.805099 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 06:33:30.809705 2991 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:30.810314 2991 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:30.815798 2991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7fc970a0e554cf98b2a14296d12f604: Generated new TSK 0
I20260929 06:33:30.815975 2991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:30.826473 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 2911
2026-09-29T06:33:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:30 dist-test-slave-sm0p krb5kdc[3010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:30 dist-test-slave-sm0p krb5kdc[3010](info): set up 2 sockets
Sep 29 06:33:30 dist-test-slave-sm0p krb5kdc[3010](info): commencing operation
krb5kdc: starting...
W20260929 06:33:31.830259 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:31 dist-test-slave-sm0p krb5kdc[3010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:31.942628 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45523
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:46015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:32.019936 3023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:32.020138 3023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:32.020159 3023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:32.021212 3023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:32.021245 3023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:32.021258 3023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:32.021270 3023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:32.021282 3023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:32.022634 3023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:46015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:32.022814 3023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:32.022979 3023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:32.027798 3029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:32.027798 3028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:32.027805 3031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:32.028054 3023 server_base.cc:1034] running on GCE node
I20260929 06:33:32.028291 3023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:32.028561 3023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:32.029774 3023 hybrid_clock.cc:648] HybridClock initialized: now 1790663612029765 us; error 44 us; skew 500 ppm
Sep 29 06:33:32 dist-test-slave-sm0p krb5kdc[3010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:32.039088 3023 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:32.039523 3023 webserver.cc:460] Webserver started at http://127.13.15.190:38651/ using document root <none> and password file <none>
I20260929 06:33:32.039770 3023 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:32.039837 3023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:32.039986 3023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:32.041051 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "f2e63afda5d34b8a834b2922bfcaa186"
format_stamp: "Formatted at 2026-09-29 06:33:32 on dist-test-slave-sm0p"
server_key: "1542f7f19386fca72f5367f40803b0af"
server_key_iv: "49bb4668f2ae3956ab10c3c135205145"
server_key_version: "encryptionkey@0"
I20260929 06:33:32.041406 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "f2e63afda5d34b8a834b2922bfcaa186"
format_stamp: "Formatted at 2026-09-29 06:33:32 on dist-test-slave-sm0p"
server_key: "1542f7f19386fca72f5367f40803b0af"
server_key_iv: "49bb4668f2ae3956ab10c3c135205145"
server_key_version: "encryptionkey@0"
I20260929 06:33:32.042811 3023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:32.043535 3038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:32.043743 3023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 06:33:32.043836 3023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "f2e63afda5d34b8a834b2922bfcaa186"
format_stamp: "Formatted at 2026-09-29 06:33:32 on dist-test-slave-sm0p"
server_key: "1542f7f19386fca72f5367f40803b0af"
server_key_iv: "49bb4668f2ae3956ab10c3c135205145"
server_key_version: "encryptionkey@0"
I20260929 06:33:32.043889 3023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:32.047294 3023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:32.049070 3023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:32.049173 3023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:32.053180 3023 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:45523
I20260929 06:33:32.053207 3090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:45523 every 8 connection(s)
I20260929 06:33:32.053545 3023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:32.054656 3091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:32.056743 3091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186: Bootstrap starting.
I20260929 06:33:32.057011 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3023
I20260929 06:33:32.057139 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:32.057333 13374 external_mini_cluster.cc:1428] Setting key 3f68dddbb9acd68d05794dde22299a85
I20260929 06:33:32.057430 3091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:32.057765 3091 log.cc:824] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:32.058509 3091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186: No bootstrap required, opened a new log
I20260929 06:33:32.059837 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:32.057888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46327 (local address 127.13.15.190:45523)
0929 06:33:32.058071 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:32.058079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:32.058108 (+ 29us) server_negotiation.cc:407] Connection header received
0929 06:33:32.058548 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:32.058554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:32.058577 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:32.058762 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:32.059444 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:32.059447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:32.059448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:32.059452 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:32.059498 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:32.059508 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:32.059602 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:32.059641 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 06:33:32.059681 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260929 06:33:32.060865 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:32.061012 3091 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } }
I20260929 06:33:32.061173 3091 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 06:33:32.061223 3091 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e63afda5d34b8a834b2922bfcaa186, State: Initialized, Role: FOLLOWER
I20260929 06:33:32.061335 3091 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } }
I20260929 06:33:32.061414 3091 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 06:33:32.061437 3091 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 06:33:32.061504 3091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:32.062120 3091 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } }
I20260929 06:33:32.062264 3091 leader_election.cc:302] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e63afda5d34b8a834b2922bfcaa186; no voters:
I20260929 06:33:32.062427 3091 leader_election.cc:288] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 06:33:32.062462 3101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:32.062638 3101 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [term 1 LEADER]: Becoming Leader. State: Replica: f2e63afda5d34b8a834b2922bfcaa186, State: Running, Role: LEADER
I20260929 06:33:32.062690 3091 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:32.062781 3101 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } }
I20260929 06:33:32.063129 3101 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e63afda5d34b8a834b2922bfcaa186. Latest consensus state: current_term: 1 leader_uuid: "f2e63afda5d34b8a834b2922bfcaa186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } } }
I20260929 06:33:32.063138 3103 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2e63afda5d34b8a834b2922bfcaa186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e63afda5d34b8a834b2922bfcaa186" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45523 } } }
I20260929 06:33:32.063297 3101 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:32.063360 3103 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186 [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:32.063637 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:32.062312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57845 (local address 127.13.15.190:45523)
0929 06:33:32.062379 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:32.062381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:32.062492 (+ 111us) server_negotiation.cc:407] Connection header received
0929 06:33:32.062603 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:32.062604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:32.062609 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:32.062667 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:32.063349 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:32.063351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:32.063351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 06:33:32.063356 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:32.063370 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 06:33:32.063374 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:32.063424 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:32.063553 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 06:33:32.063567 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:33:32.063663 3108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:32.064188 3108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:32.067265 3108 catalog_manager.cc:1347] Generated new cluster ID: 8b69add741004364903a1dc71745a6a7
I20260929 06:33:32.067325 3108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:32.084741 3108 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:32.084967 3108 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:32.092648 3108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2e63afda5d34b8a834b2922bfcaa186: Generated new TSK 0
I20260929 06:33:32.092868 3108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 06:33:32.098755 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3023
2026-09-29T06:33:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1272 ms)
[----------] 18 tests from AuthTokenIssuingTest (23099 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:32 dist-test-slave-sm0p krb5kdc[3122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:32 dist-test-slave-sm0p krb5kdc[3122](info): set up 2 sockets
Sep 29 06:33:32 dist-test-slave-sm0p krb5kdc[3122](info): commencing operation
krb5kdc: starting...
W20260929 06:33:33.142493 13374 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:33.260124 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35633
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:33.338271 3135 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:33.338465 3135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:33.338482 3135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:33.338501 3135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:33.339546 3135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:33.339579 3135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:33.339593 3135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:33.339605 3135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:33.339618 3135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:33.341002 3135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:33.341176 3135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:33.341343 3135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:33.346062 3140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:33.346040 3143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:33.346083 3135 server_base.cc:1034] running on GCE node
W20260929 06:33:33.346055 3141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:33.346483 3135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:33.346715 3135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:33.347903 3135 hybrid_clock.cc:648] HybridClock initialized: now 1790663613347888 us; error 43 us; skew 500 ppm
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:33.350252 3135 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:33.350656 3135 webserver.cc:460] Webserver started at http://127.13.15.190:34641/ using document root <none> and password file <none>
I20260929 06:33:33.350907 3135 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:33.350975 3135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:33.351119 3135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:33.352133 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "c079fb2786f94df8af6c1e54f913fe2b"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "17c9f7afe117689c09f41420637b3fb1"
server_key_iv: "41c5b15d1bff44bb76ba7554cbc55fb4"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.352494 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "c079fb2786f94df8af6c1e54f913fe2b"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "17c9f7afe117689c09f41420637b3fb1"
server_key_iv: "41c5b15d1bff44bb76ba7554cbc55fb4"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.353814 3135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 06:33:33.354614 3150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:33.354796 3135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:33.354907 3135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "c079fb2786f94df8af6c1e54f913fe2b"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "17c9f7afe117689c09f41420637b3fb1"
server_key_iv: "41c5b15d1bff44bb76ba7554cbc55fb4"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.354979 3135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:33.363513 3135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:33.365289 3135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:33.365447 3135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:33.369107 3135 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:35633
I20260929 06:33:33.369163 3202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:35633 every 8 connection(s)
I20260929 06:33:33.369488 3135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:33.370431 3203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:33.371762 3203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.374683 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3135
I20260929 06:33:33.374770 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:33.374962 13374 external_mini_cluster.cc:1428] Setting key 3de3dd85cb3d42b623de3e0a4951159b
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:33.381201 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.372326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35633 (local address 127.0.0.1:51740)
0929 06:33:33.372544 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.372584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.373051 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.373354 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.373357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.373623 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.373776 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.373780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.374879 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.374879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.375583 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.375585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.376146 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.376148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.376224 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.376706 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.376713 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.378044 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.379804 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.379810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.379812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.379991 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.380226 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.380227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.380227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.380317 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.380643 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.380648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.380703 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.380800 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.381019 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":93,"thread_start_us":96,"threads_started":1}
I20260929 06:33:33.381203 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.372402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.13.15.190:35633)
0929 06:33:33.372552 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.372561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.372681 (+ 120us) server_negotiation.cc:407] Connection header received
0929 06:33:33.373220 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.373222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.373239 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.373350 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.373873 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.374766 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.375679 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.376017 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.376133 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.378130 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.378160 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.378161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.378168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.379681 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.380083 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.380086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.380086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.380162 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.380381 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.380382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.380383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.380576 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.380663 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.380836 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.380958 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.189@KRBTEST.COM; defaulting to no policy
I20260929 06:33:33.381907 3203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.382263 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.382127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48084)
0929 06:33:33.382198 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.382244 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 06:33:33.382395 3153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
I20260929 06:33:33.382833 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.382704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48088)
0929 06:33:33.382787 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.382814 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260929 06:33:33.382952 3203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37455: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
I20260929 06:33:33.396845 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37455
--webserver_interface=127.13.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:33:33.432149 3203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } attempt: 1
I20260929 06:33:33.432698 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.432567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48090)
0929 06:33:33.432632 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.432673 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 06:33:33.433100 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.433021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48098)
0929 06:33:33.433062 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.433085 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:33:33.433223 3203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37455: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
W20260929 06:33:33.476706 3210 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:33.476923 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:33.476940 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:33.476960 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:33.478001 3210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:33.478035 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:33.478049 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:33.478062 3210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:33.478075 3210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:33.479565 3210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:33.479756 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:33.479911 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:33.484894 3215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:33.484982 3216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:33.484908 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:33.485005 3210 server_base.cc:1034] running on GCE node
I20260929 06:33:33.485319 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:33.485550 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:33.486716 3210 hybrid_clock.cc:648] HybridClock initialized: now 1790663613486689 us; error 37 us; skew 500 ppm
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:33.489152 3210 init.cc:377] Logged in from keytab as kudu/127.13.15.189@KRBTEST.COM (short username kudu)
I20260929 06:33:33.489599 3210 webserver.cc:460] Webserver started at http://127.13.15.189:34963/ using document root <none> and password file <none>
I20260929 06:33:33.489828 3210 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:33.489902 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:33.490043 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:33.491204 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data/instance:
uuid: "68185becc5234a52bebfa75d3758437d"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "9c80c00d7783c2fee0291d5d5d0e47c2"
server_key_iv: "cd2151a95b82edef04f49e220cda1651"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.491562 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal/instance:
uuid: "68185becc5234a52bebfa75d3758437d"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "9c80c00d7783c2fee0291d5d5d0e47c2"
server_key_iv: "cd2151a95b82edef04f49e220cda1651"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.492904 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 06:33:33.493562 3225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:33.493719 3210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:33.493818 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "68185becc5234a52bebfa75d3758437d"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "9c80c00d7783c2fee0291d5d5d0e47c2"
server_key_iv: "cd2151a95b82edef04f49e220cda1651"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.493898 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:33.501387 3203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } attempt: 2
I20260929 06:33:33.501884 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.501750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48110)
0929 06:33:33.501814 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.501854 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 06:33:33.502293 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.502207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48114)
0929 06:33:33.502250 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.502276 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260929 06:33:33.502405 3203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37455: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37455: connect: Connection refused (error 111)
I20260929 06:33:33.509776 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:33.511816 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:33.511931 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:33.516095 3210 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.189:37455
I20260929 06:33:33.516201 3277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.189:37455 every 8 connection(s)
I20260929 06:33:33.516510 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
I20260929 06:33:33.517495 3278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:33.519146 3278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.521196 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3210
I20260929 06:33:33.521296 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-1/wal/instance
I20260929 06:33:33.521497 13374 external_mini_cluster.cc:1428] Setting key b6aaea275da9e8d4ca03377777246de8
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:33.528672 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.519823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.13.15.190:35633)
0929 06:33:33.519880 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.519882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.520080 (+ 198us) server_negotiation.cc:407] Connection header received
0929 06:33:33.520500 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.520501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.520509 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.520556 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.521152 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.521930 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.523024 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.523242 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.523329 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.525702 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.525708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.525708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.525727 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.527253 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.527696 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.527698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.527698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.527755 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.528018 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.528019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.528020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.528127 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.528199 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.528501 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.528585 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 06:33:33.528831 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.519790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35633 (local address 127.0.0.1:51744)
0929 06:33:33.519963 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.519998 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.520417 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.520625 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.520628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.520867 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.521066 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.521070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.522043 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.522043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.522888 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.522891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.523357 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.523359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.523487 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.524024 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.524034 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.525592 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.527349 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.527356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.527358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.527606 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.527826 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.527827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.527828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.527952 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.528200 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.528207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.528356 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.528469 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.528611 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.13.15.188@KRBTEST.COM; defaulting to no policy
I20260929 06:33:33.529433 3278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.188@KRBTEST.COM" created.
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:33.537364 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.529660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48130)
0929 06:33:33.529705 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.529714 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.529747 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.530044 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.530045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.530250 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.530317 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.530319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.531434 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.531435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.532176 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.532179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.532825 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.532826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.532895 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.533277 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.533281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.534542 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.536294 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.536295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.536295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.536459 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.536676 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.536677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.536677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.536729 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.537017 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.537018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.537055 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.537117 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.537212 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 06:33:33.537403 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.529742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.13.15.189:37455)
0929 06:33:33.529875 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.529879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.529900 (+ 21us) server_negotiation.cc:407] Connection header received
0929 06:33:33.529915 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.529916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.529926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.530004 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.530411 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.531343 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.532264 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.532745 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.532827 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.534646 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.534653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.534654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.534660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.536177 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.536550 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.536553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.536554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.536614 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.536798 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.536799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.536799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.536949 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.537038 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.537142 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.537271 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260929 06:33:33.538272 3278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.538604 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.538502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44620)
0929 06:33:33.538543 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.538583 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260929 06:33:33.538775 3229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
I20260929 06:33:33.539189 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.539106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44632)
0929 06:33:33.539152 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.539174 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 06:33:33.539302 3278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34657: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
I20260929 06:33:33.550299 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34657
--webserver_interface=127.13.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:40367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:33:33.588541 3278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } attempt: 1
I20260929 06:33:33.589073 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.588930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44646)
0929 06:33:33.588991 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.589046 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 06:33:33.589442 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.589355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44660)
0929 06:33:33.589401 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.589424 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 06:33:33.589546 3278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34657: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
I20260929 06:33:33.593552 3203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } attempt: 3
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:33.600498 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.593816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48134)
0929 06:33:33.593874 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.593899 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.593953 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.594118 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.594119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.594299 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.594359 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.594361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.595039 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.595041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.595577 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.595579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.595972 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.595973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.596052 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.596563 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.596567 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.597753 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.599426 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.599427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.599427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.599658 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.599917 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.599918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.599919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.599985 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.600255 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.600255 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.600316 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.600367 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.600438 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 06:33:33.600548 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.593868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.13.15.189:37455)
0929 06:33:33.593922 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.593924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.593937 (+ 13us) server_negotiation.cc:407] Connection header received
0929 06:33:33.594027 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.594028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.594041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.594112 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.594451 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.594953 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.595662 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.595897 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.595974 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.597829 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.597831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.597831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.597839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.599309 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.599770 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.599771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.599772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.599834 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.600062 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.600063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.600064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.600188 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.600255 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.600392 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.600466 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 06:33:33.601043 3203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.601362 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.601265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44672)
0929 06:33:33.601310 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.601338 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 06:33:33.601673 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.601589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44688)
0929 06:33:33.601630 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.601653 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:33:33.601763 3203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34657: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
W20260929 06:33:33.629128 3285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:33.629360 3285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:33.629390 3285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:33.629429 3285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:33.630551 3285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:33.630612 3285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:33.630636 3285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:33.630662 3285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:33.630692 3285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:33.632133 3285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:40367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:35633,127.13.15.189:37455,127.13.15.188:34657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:33.632403 3285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:33.632627 3285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 06:33:33.632905 3203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } attempt: 1
I20260929 06:33:33.633289 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.633175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44704)
0929 06:33:33.633228 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.633265 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 06:33:33.633584 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.633497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44714)
0929 06:33:33.633539 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.633561 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:33:33.633797 3203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34657: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
W20260929 06:33:33.637372 3291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:33.637374 3293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:33.637365 3290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:33.637666 3285 server_base.cc:1034] running on GCE node
I20260929 06:33:33.637889 3285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:33.638201 3285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:33.639372 3285 hybrid_clock.cc:648] HybridClock initialized: now 1790663613639349 us; error 33 us; skew 500 ppm
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:33.641508 3285 init.cc:377] Logged in from keytab as kudu/127.13.15.188@KRBTEST.COM (short username kudu)
I20260929 06:33:33.641940 3285 webserver.cc:460] Webserver started at http://127.13.15.188:44131/ using document root <none> and password file <none>
I20260929 06:33:33.642216 3285 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:33.642293 3285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:33.642419 3285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:33.643352 3285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data/instance:
uuid: "7d78328c0a2c44f8b03f4b8119bbbca6"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "3936e0978e4cc023593f295fa4501c9a"
server_key_iv: "e2a7ab478e9b7bf490f1a0d65be0ab70"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.643764 3285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal/instance:
uuid: "7d78328c0a2c44f8b03f4b8119bbbca6"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "3936e0978e4cc023593f295fa4501c9a"
server_key_iv: "e2a7ab478e9b7bf490f1a0d65be0ab70"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.645081 3285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 06:33:33.645749 3300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:33.645915 3285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 06:33:33.646004 3285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "7d78328c0a2c44f8b03f4b8119bbbca6"
format_stamp: "Formatted at 2026-09-29 06:33:33 on dist-test-slave-sm0p"
server_key: "3936e0978e4cc023593f295fa4501c9a"
server_key_iv: "e2a7ab478e9b7bf490f1a0d65be0ab70"
server_key_version: "encryptionkey@0"
I20260929 06:33:33.646083 3285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:33.657449 3285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:33.657652 3278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } attempt: 2
I20260929 06:33:33.658066 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.657940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44728)
0929 06:33:33.658001 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.658038 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 06:33:33.658435 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.658347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44738)
0929 06:33:33.658394 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.658419 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:33:33.658545 3278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34657: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34657: connect: Connection refused (error 111)
I20260929 06:33:33.659888 3285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:33.659991 3285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:33.663640 3285 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.188:34657
I20260929 06:33:33.663729 3352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.188:34657 every 8 connection(s)
I20260929 06:33:33.664040 3285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
I20260929 06:33:33.664678 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3285
I20260929 06:33:33.664786 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790663323716959-13374-0/minicluster-data/master-2/wal/instance
I20260929 06:33:33.664999 13374 external_mini_cluster.cc:1428] Setting key 131ccabda466ea09731503758e7a36b0
I20260929 06:33:33.665041 3353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:33.667032 3353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:33.675194 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.667688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51760 (local address 127.13.15.190:35633)
0929 06:33:33.667737 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.667738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.667843 (+ 105us) server_negotiation.cc:407] Connection header received
0929 06:33:33.667941 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.667943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.667953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.668027 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.668441 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.668906 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.669655 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.669896 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.669966 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.672467 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.672469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.672469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.672476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.673906 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.674322 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.674323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.674324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.674389 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.674623 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.674623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.674624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.674743 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.674817 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.675021 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.675104 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 06:33:33.675407 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.665513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.13.15.188:34657)
0929 06:33:33.665696 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.665715 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.665742 (+ 27us) server_negotiation.cc:407] Connection header received
0929 06:33:33.666100 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.666106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.666205 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.666353 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.666980 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.668077 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.668826 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.669290 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.669406 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.671967 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.671974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.671974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.671981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.673724 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.674119 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.674121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.674121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.674272 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.674506 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.674507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.674507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.674772 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.674916 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.675020 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.675130 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260929 06:33:33.675431 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.667596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:35633 (local address 127.0.0.1:51760)
0929 06:33:33.667737 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.667781 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.667864 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.668046 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.668049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.668294 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.668352 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.668353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.668994 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.668995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.669557 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.669558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.670004 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.670006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.670082 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.670525 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.670535 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.672386 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.673983 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.673989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.673992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.674254 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.674457 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.674458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.674459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.674552 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.674830 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.674837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.674899 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.674993 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.675325 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":71,"thread_start_us":83,"threads_started":1}
I20260929 06:33:33.675916 3353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:33.682539 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.676129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37455 (local address 127.0.0.1:48150)
0929 06:33:33.676165 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.676175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.676208 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.676331 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.676331 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.676458 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.676497 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.676498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.677165 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.677166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.677632 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.677634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.678001 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.678002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.678066 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.678474 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.678477 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.679737 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.681283 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.681284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.681284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.681490 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.681911 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.681912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.681912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.681957 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.682239 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.682240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.682279 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.682368 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.682451 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 06:33:33.682539 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.676190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.13.15.189:37455)
0929 06:33:33.676230 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.676232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.676238 (+ 6us) server_negotiation.cc:407] Connection header received
0929 06:33:33.676264 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.676265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.676273 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.676327 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.676621 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.677086 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.677722 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.677931 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.678005 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.679817 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.679819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.679819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.679826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.681185 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.681547 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.681548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.681548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.681625 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.682034 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.682035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.682036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.682180 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.682239 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.682392 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.682456 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:33.682861 3353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:33.683828 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.676210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.13.15.190:35633)
0929 06:33:33.676244 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.676246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.676254 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:33.676322 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.676322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.676331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.676384 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.676844 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.677270 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.677947 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.678186 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.678248 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.681301 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.681302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.681302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.681309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.682679 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.683044 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.683045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.683046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.683101 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.683286 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.683287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.683287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.683436 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.683548 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.683654 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.683724 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:33.689985 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.683069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44758)
0929 06:33:33.683174 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.683184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.683225 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.683361 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.683361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.683490 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.683519 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.683520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.684417 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.684418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.685196 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.685198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.685632 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.685632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.685666 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.685991 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.685996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.687386 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.688896 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.688897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.688898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.689102 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.689292 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.689292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.689292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.689341 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.689609 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.689610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.689650 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.689724 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.689811 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 06:33:33.689996 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.683102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.13.15.188:34657)
0929 06:33:33.683132 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.683134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.683213 (+ 79us) server_negotiation.cc:407] Connection header received
0929 06:33:33.683286 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.683286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.683294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.683356 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.683593 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.684343 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.685262 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.685563 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.685629 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.687465 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.687467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.687467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.687474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.688811 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.689159 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.689160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.689160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.689211 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.689401 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.689402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.689403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.689530 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.689607 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.689769 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.689869 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":15}
I20260929 06:33:33.691591 3353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: Bootstrap starting.
I20260929 06:33:33.692051 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.684490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.13.15.189:37455)
0929 06:33:33.684522 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.684523 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.684529 (+ 6us) server_negotiation.cc:407] Connection header received
0929 06:33:33.684549 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.684550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.684558 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.684602 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.685121 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.685730 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.686629 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.686945 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.687016 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.689538 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.689539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.689539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.689544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.691052 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.691423 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.691423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.691423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.691466 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.691643 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.691643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.691643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.691755 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.691849 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.691952 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.692001 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 06:33:33.692541 3353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:33.692871 3353 log.cc:824] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:33.693428 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:33.693655 3353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: No bootstrap required, opened a new log
I20260929 06:33:33.694526 3177 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c079fb2786f94df8af6c1e54f913fe2b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51764
I20260929 06:33:33.695129 3353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.695288 3353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d78328c0a2c44f8b03f4b8119bbbca6, State: Initialized, Role: FOLLOWER
I20260929 06:33:33.695374 3353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.695665 3359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.695706 3353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:33.695801 3359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 06:33:33.697499 3370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:33:33.697576 3370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:33.697798 3285 master_runner.cc:186] Error getting master registration for 127.13.15.190:35633: OK, instance_id { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" instance_seqno: 1790663613368128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:33:33.700562 3285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:33:33.708961 3203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } attempt: 2
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:33:33.715420 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.709241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44772)
0929 06:33:33.709290 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.709301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.709344 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.709471 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.709471 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.709627 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.709664 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.709666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.710358 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.710359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.710825 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.710827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.711167 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.711168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.711219 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.711633 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.711637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.712723 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.714350 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.714351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.714351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.714552 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.714768 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.714768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.714769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.714829 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.715123 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.715124 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.715167 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.715232 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.715320 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 06:33:33.715435 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.709270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.13.15.188:34657)
0929 06:33:33.709332 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.709334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.709345 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:33:33.709416 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.709417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.709425 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.709469 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.709744 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.710272 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.710905 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.711100 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.711169 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.712813 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.712814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.712815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.712821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.714268 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.714637 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.714638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.714638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.714697 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.714899 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.714900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.714900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.715055 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.715127 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.715276 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.715350 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 06:33:33.717202 3203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: Bootstrap starting.
I20260929 06:33:33.717875 3203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:33.718199 3203 log.cc:824] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:33.718925 3203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: No bootstrap required, opened a new log
I20260929 06:33:33.719866 3203 raft_consensus.cc:348] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.719986 3203 raft_consensus.cc:729] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c079fb2786f94df8af6c1e54f913fe2b, State: Initialized, Role: FOLLOWER
I20260929 06:33:33.720082 3203 consensus_queue.cc:260] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.720345 3373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.720399 3203 sys_catalog.cc:564] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:33.720413 3373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.720504 3177 raft_consensus.cc:482] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 06:33:33.720606 3177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:33.721342 3177 raft_consensus.cc:504] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.721679 3177 leader_election.cc:288] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [CANDIDATE]: Term 1 election: Requested vote from peers 68185becc5234a52bebfa75d3758437d (127.13.15.189:37455), 7d78328c0a2c44f8b03f4b8119bbbca6 (127.13.15.188:34657)
W20260929 06:33:33.722158 3384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:33:33.722211 3384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:33.722596 3327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c079fb2786f94df8af6c1e54f913fe2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6"
I20260929 06:33:33.722704 3327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:33.723909 3327 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c079fb2786f94df8af6c1e54f913fe2b in term 1.
I20260929 06:33:33.724885 3154 leader_election.cc:302] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d78328c0a2c44f8b03f4b8119bbbca6, c079fb2786f94df8af6c1e54f913fe2b; no voters:
I20260929 06:33:33.725052 3373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:33.725206 3373 raft_consensus.cc:686] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [term 1 LEADER]: Becoming Leader. State: Replica: c079fb2786f94df8af6c1e54f913fe2b, State: Running, Role: LEADER
I20260929 06:33:33.725303 3373 consensus_queue.cc:237] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.725703 3373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c079fb2786f94df8af6c1e54f913fe2b. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.725804 3373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:33.726454 3135 master_runner.cc:186] Error getting master registration for 127.13.15.189:37455: OK, instance_id { permanent_uuid: "68185becc5234a52bebfa75d3758437d" instance_seqno: 1790663613514928 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:33:33.728899 3135 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:33:33.729543 3394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:33.730068 3394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:33.733496 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.725121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.13.15.190:35633)
0929 06:33:33.725160 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.725162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.725604 (+ 442us) server_negotiation.cc:407] Connection header received
0929 06:33:33.725823 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.725824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.725835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.725868 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.726676 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.727427 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.728463 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.728701 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.728744 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.730362 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.730363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.730363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.730369 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.731942 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.732283 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.732284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.732284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.732338 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.732505 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.732506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.732506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.732639 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.732723 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.733344 (+ 621us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.733413 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:33.740434 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.725486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.13.15.188:34657)
0929 06:33:33.725525 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.725527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.732361 (+ 6834us) server_negotiation.cc:407] Connection header received
0929 06:33:33.732588 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.732589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.732599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.732625 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.733166 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.733833 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.735584 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.735881 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.735917 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.737302 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.737303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.737304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.737311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.739082 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.739559 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.739560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.739560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.739615 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.739826 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.739827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.739828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.739959 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.740035 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.740275 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.740340 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:33.741730 3327 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [term 1 FOLLOWER]: Refusing update from remote peer c079fb2786f94df8af6c1e54f913fe2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:33.742087 3373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:33.743389 3359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c079fb2786f94df8af6c1e54f913fe2b. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.743510 3359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.743734 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.729109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.13.15.189:37455)
0929 06:33:33.729151 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.729153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.734075 (+ 4922us) server_negotiation.cc:407] Connection header received
0929 06:33:33.734116 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.734117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.734126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.734184 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.735529 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.736184 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.737573 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.738444 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.738492 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.740586 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.740587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.740587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.740593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.742368 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.742874 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.742874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.742875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.742925 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.743121 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.743122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.743140 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.743263 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.743350 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.743541 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.743616 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:33.749711 3278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } attempt: 3
I20260929 06:33:33.751832 3390 sys_catalog.cc:455] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.751922 3390 sys_catalog.cc:458] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:33.752362 3359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.752441 3359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.752835 3396 mvcc.cc:205] Tried to move back new op lower bound from 7334558161883303936 to 7334558161819570176. Current Snapshot: MvccSnapshot[applied={T|T < 7334558161883303936}]
I20260929 06:33:33.753589 3397 mvcc.cc:205] Tried to move back new op lower bound from 7334558161883303936 to 7334558161819570176. Current Snapshot: MvccSnapshot[applied={T|T < 7334558161883303936}]
Sep 29 06:33:33 dist-test-slave-sm0p krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:33:33.756654 3394 catalog_manager.cc:1347] Generated new cluster ID: 8c73c488cea6475c8c3815a51d6a8903
I20260929 06:33:33.756762 3394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:33.758263 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.749953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34657 (local address 127.0.0.1:44790)
0929 06:33:33.749998 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:33.750009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:33.750051 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:33.750303 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:33.750304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:33.750465 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:33.750505 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.750506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.751446 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.751448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.752219 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:33.752222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.752946 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.752948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:33.753006 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.753714 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:33.753719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:33.755360 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:33.757245 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.757246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.757247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.757438 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.757654 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:33.757655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:33.757655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:33.757710 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:33.757955 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:33.757956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:33.757994 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 06:33:33.758049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 06:33:33.758129 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 06:33:33.758651 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:33.749994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.13.15.188:34657)
0929 06:33:33.750028 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.750030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.750037 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:33:33.750127 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.750201 (+ 74us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.750220 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.750273 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.750579 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.751347 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.752401 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:33.752723 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:33.752801 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:33.755444 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:33.755446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:33.755446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:33.755453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:33.757157 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.757528 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.757529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.757530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.757587 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:33.757769 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:33.757769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:33.757770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:33.757881 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:33.757934 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:33.758477 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 06:33:33.758548 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:33:33.760320 3278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: Bootstrap starting.
I20260929 06:33:33.760995 3278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:33.761300 3278 log.cc:824] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:33.762009 3278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: No bootstrap required, opened a new log
I20260929 06:33:33.763741 3278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.763896 3278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68185becc5234a52bebfa75d3758437d, State: Initialized, Role: FOLLOWER
I20260929 06:33:33.763993 3278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } }
I20260929 06:33:33.764298 3402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.764370 3278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:33.764441 3251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:33.764472 3402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.765375 3251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [term 1 FOLLOWER]: Refusing update from remote peer c079fb2786f94df8af6c1e54f913fe2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:33.765564 3252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c079fb2786f94df8af6c1e54f913fe2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68185becc5234a52bebfa75d3758437d"
I20260929 06:33:33.766783 3252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c079fb2786f94df8af6c1e54f913fe2b in term 1.
I20260929 06:33:33.767091 3373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:33.769929 3402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c079fb2786f94df8af6c1e54f913fe2b. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.770016 3402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.770291 3416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.770407 3416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:33.778811 3413 catalog_manager.cc:1259] Loaded cluster ID: 8c73c488cea6475c8c3815a51d6a8903
I20260929 06:33:33.778882 3413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: loading cluster ID for follower catalog manager: success
W20260929 06:33:33.779026 3413 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260929 06:33:33.779124 3413 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:33.815179 3373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c079fb2786f94df8af6c1e54f913fe2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c079fb2786f94df8af6c1e54f913fe2b" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 35633 } } peers { permanent_uuid: "68185becc5234a52bebfa75d3758437d" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37455 } } peers { permanent_uuid: "7d78328c0a2c44f8b03f4b8119bbbca6" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34657 } } }
I20260929 06:33:33.815299 3373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:33.817569 3394 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:33.818382 3394 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:33.826210 3394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c079fb2786f94df8af6c1e54f913fe2b: Generated new TSK 0
I20260929 06:33:33.826385 3394 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:33:33.850097 3356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.848559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.13.15.188:34657)
0929 06:33:33.848606 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.848608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.849344 (+ 736us) server_negotiation.cc:407] Connection header received
0929 06:33:33.849481 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.849482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.849493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.849523 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.850032 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44792: BlockingRecv error: recv got EOF from 127.0.0.1:44792 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 06:33:33.850911 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.849000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.13.15.189:37455)
0929 06:33:33.849040 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.849043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.850354 (+ 1311us) server_negotiation.cc:407] Connection header received
0929 06:33:33.850456 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.850457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.850468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.850500 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.850847 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48160: BlockingRecv error: recv got EOF from 127.0.0.1:48160 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 06:33:33.852020 3206 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.851326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.13.15.190:35633)
0929 06:33:33.851366 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.851368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.851488 (+ 120us) server_negotiation.cc:407] Connection header received
0929 06:33:33.851582 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.851583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.851593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.851622 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.851964 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51786: BlockingRecv error: recv got EOF from 127.0.0.1:51786 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 06:33:33.854339 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.853564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48162 (local address 127.13.15.189:37455)
0929 06:33:33.853603 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.853605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.853821 (+ 216us) server_negotiation.cc:407] Connection header received
0929 06:33:33.853924 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.853925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.853942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.853970 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.854297 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48162: BlockingRecv error: recv got EOF from 127.0.0.1:48162 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 06:33:33.857715 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.856704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.13.15.189:37455)
0929 06:33:33.856741 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.856743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.857064 (+ 321us) server_negotiation.cc:407] Connection header received
0929 06:33:33.857173 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.857174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.857203 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.857236 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.857656 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48172: BlockingRecv error: recv got EOF from 127.0.0.1:48172 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 06:33:33.861734 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.860853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.13.15.189:37455)
0929 06:33:33.860889 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.860891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.861162 (+ 271us) server_negotiation.cc:407] Connection header received
0929 06:33:33.861266 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.861267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.861277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.861309 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.861686 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48178: BlockingRecv error: recv got EOF from 127.0.0.1:48178 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 06:33:33.867604 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.866827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.13.15.189:37455)
0929 06:33:33.866863 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.866864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.867107 (+ 243us) server_negotiation.cc:407] Connection header received
0929 06:33:33.867208 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.867209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.867219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.867248 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.867564 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48180: BlockingRecv error: recv got EOF from 127.0.0.1:48180 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 06:33:33.877364 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.876623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.13.15.189:37455)
0929 06:33:33.876653 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.876655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.876899 (+ 244us) server_negotiation.cc:407] Connection header received
0929 06:33:33.876983 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.876984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.876993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.877019 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.877312 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48196: BlockingRecv error: recv got EOF from 127.0.0.1:48196 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 06:33:33.895437 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.894684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.13.15.189:37455)
0929 06:33:33.894728 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.894731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.894873 (+ 142us) server_negotiation.cc:407] Connection header received
0929 06:33:33.894957 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.894957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.894969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.895044 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.895370 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48212: BlockingRecv error: recv got EOF from 127.0.0.1:48212 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 06:33:33.929528 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.928721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.13.15.189:37455)
0929 06:33:33.928801 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.928804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.928931 (+ 127us) server_negotiation.cc:407] Connection header received
0929 06:33:33.929003 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.929004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.929016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.929091 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.929462 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48214: BlockingRecv error: recv got EOF from 127.0.0.1:48214 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 06:33:33.995654 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:33.994824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.13.15.189:37455)
0929 06:33:33.994926 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:33.994929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:33.995036 (+ 107us) server_negotiation.cc:407] Connection header received
0929 06:33:33.995138 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:33.995139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:33.995150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:33.995227 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:33.995579 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48216: BlockingRecv error: recv got EOF from 127.0.0.1:48216 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 06:33:34.125581 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:34.124836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.13.15.189:37455)
0929 06:33:34.124913 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.124915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.125042 (+ 127us) server_negotiation.cc:407] Connection header received
0929 06:33:34.125098 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.125099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.125109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.125151 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:34.125495 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48218: BlockingRecv error: recv got EOF from 127.0.0.1:48218 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 06:33:34.383373 3282 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:34.382731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.13.15.189:37455)
0929 06:33:34.382833 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.382837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.382849 (+ 12us) server_negotiation.cc:407] Connection header received
0929 06:33:34.382872 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.382872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.382882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.382937 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:34.383291 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48228: BlockingRecv error: recv got EOF from 127.0.0.1:48228 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 06:33:34.698009 3370 catalog_manager.cc:1259] Loaded cluster ID: 8c73c488cea6475c8c3815a51d6a8903
I20260929 06:33:34.698097 3370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: loading cluster ID for follower catalog manager: success
I20260929 06:33:34.698882 3370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: acquiring CA information for follower catalog manager: success
I20260929 06:33:34.699191 3370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d78328c0a2c44f8b03f4b8119bbbca6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:33:34.780289 3413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: acquiring CA information for follower catalog manager: success
I20260929 06:33:34.780601 3413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68185becc5234a52bebfa75d3758437d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:33:34.899338 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:34.896558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.13.15.189:37455)
0929 06:33:34.896680 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.896683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.896739 (+ 56us) server_negotiation.cc:407] Connection header received
0929 06:33:34.896815 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.896816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.896884 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.896982 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:34.897111 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.897668 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.898496 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.898696 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.898804 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:34.898990 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:34.899094 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:34.899176 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:34.899215 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 06:33:34.899236 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260929 06:33:34.903560 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:34.901075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.13.15.188:34657)
0929 06:33:34.901173 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.901175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.901183 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:34.901256 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.901257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.901302 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.901390 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:34.901481 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.901946 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.902712 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.902929 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.903011 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:34.903193 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:34.903300 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:34.903386 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:34.903443 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:33:34.903463 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":50,"threads_started":1}
I20260929 06:33:34.907917 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:34.905005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.13.15.189:37455)
0929 06:33:34.905042 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.905044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.905286 (+ 242us) server_negotiation.cc:407] Connection header received
0929 06:33:34.905343 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.905344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.905353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.905409 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:34.905553 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.906304 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907145 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.907442 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907503 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:34.907632 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:34.907661 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:34.907709 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:34.907823 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 06:33:34.907846 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:34.907924 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:34.904991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.13.15.190:35633)
0929 06:33:34.905134 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.905137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.905166 (+ 29us) server_negotiation.cc:407] Connection header received
0929 06:33:34.905268 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.905269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.905319 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.905407 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:34.905507 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.906014 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907069 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.907347 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907424 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:34.907616 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:34.907719 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:34.907782 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:34.907831 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 06:33:34.907853 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260929 06:33:34.908077 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:34.905060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.13.15.188:34657)
0929 06:33:34.905194 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:34.905196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:34.905235 (+ 39us) server_negotiation.cc:407] Connection header received
0929 06:33:34.905345 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:34.905346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:34.905356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:34.905384 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:34.905593 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.906298 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907341 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:34.907622 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:34.907685 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:34.907816 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:34.907846 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:34.907873 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:34.907987 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 06:33:34.908010 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260929 06:33:34.909910 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3135
I20260929 06:33:34.915937 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3210
I20260929 06:33:34.920598 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3285
2026-09-29T06:33:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2823 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 06:33:34 dist-test-slave-sm0p krb5kdc[3499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 06:33:34 dist-test-slave-sm0p krb5kdc[3499](info): set up 2 sockets
Sep 29 06:33:34 dist-test-slave-sm0p krb5kdc[3499](info): commencing operation
krb5kdc: starting...
W20260929 06:33:35.918406 13374 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/test-user.keytab.
Sep 29 06:33:35 dist-test-slave-sm0p krb5kdc[3499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T06:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T06:33:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
Entry for principal kudu/127.13.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab.
I20260929 06:33:36.035066 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45203
--webserver_interface=127.13.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 06:33:36.117817 3512 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:36.118003 3512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:36.118021 3512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:36.118039 3512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:36.119211 3512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:36.119248 3512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:36.119263 3512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:36.119305 3512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:36.119323 3512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:36.120752 3512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.190.keytab
--principal=kudu/127.13.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.190:45203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:36.120992 3512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:36.121208 3512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:36.125913 3520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.125921 3517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.126225 3518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:36.126070 3512 server_base.cc:1034] running on GCE node
I20260929 06:33:36.126507 3512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:36.126770 3512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:36.127934 3512 hybrid_clock.cc:648] HybridClock initialized: now 1790663616127921 us; error 43 us; skew 500 ppm
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:36.130340 3512 init.cc:377] Logged in from keytab as kudu/127.13.15.190@KRBTEST.COM (short username kudu)
I20260929 06:33:36.130703 3512 webserver.cc:460] Webserver started at http://127.13.15.190:33943/ using document root <none> and password file <none>
I20260929 06:33:36.130916 3512 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:36.130980 3512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:36.131115 3512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:36.132079 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data/instance:
uuid: "fface9f866124414911200e23da68b6a"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "8e9633c2a1de12fd9a07fbfc2d8487cb"
server_key_iv: "0b5e8acdd7817c66ed09b4e18321ca4c"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.132460 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance:
uuid: "fface9f866124414911200e23da68b6a"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "8e9633c2a1de12fd9a07fbfc2d8487cb"
server_key_iv: "0b5e8acdd7817c66ed09b4e18321ca4c"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.133724 3512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 06:33:36.134443 3527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.134614 3512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.134708 3512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal
uuid: "fface9f866124414911200e23da68b6a"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "8e9633c2a1de12fd9a07fbfc2d8487cb"
server_key_iv: "0b5e8acdd7817c66ed09b4e18321ca4c"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.134805 3512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:36.150941 3512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:36.153131 3512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:36.153301 3512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:36.157214 3512 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.190:45203
I20260929 06:33:36.157246 3579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.190:45203 every 8 connection(s)
I20260929 06:33:36.157616 3512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/data/info.pb
I20260929 06:33:36.158640 3580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:36.159612 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3512
I20260929 06:33:36.159746 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-0/wal/instance
I20260929 06:33:36.159958 13374 external_mini_cluster.cc:1428] Setting key a4bc19e88bf438d7b02dd1d607aeade1
I20260929 06:33:36.160648 3580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.189@KRBTEST.COM" created.
I20260929 06:33:36.171442 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.161439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45203 (local address 127.0.0.1:51804)
0929 06:33:36.161719 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.161759 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.162444 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.162724 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.162727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.163099 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.163323 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.163329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.164351 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.164352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.165577 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.165580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.166129 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.166149 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.166247 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.166730 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.166740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.168144 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.169731 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.169737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.169738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.169925 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.170424 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.170424 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.170425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.170505 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.170849 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.170856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.170919 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.171027 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.171275 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":114,"spinlock_wait_cycles":6912,"thread_start_us":93,"threads_started":1}
I20260929 06:33:36.171442 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.161439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.13.15.190:45203)
0929 06:33:36.161635 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.161645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.161870 (+ 225us) server_negotiation.cc:407] Connection header received
0929 06:33:36.162590 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.162592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.162608 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.162680 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.163422 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.164257 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.165682 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.166016 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.166098 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.168267 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.168274 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.168274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.168281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.169652 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.170252 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.170255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.170255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.170358 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.170570 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.170570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.170570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.170762 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.170824 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.171064 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.171178 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260929 06:33:36.172128 3580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.172493 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.172365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60284)
0929 06:33:36.172412 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.172462 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 06:33:36.172609 3529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
I20260929 06:33:36.173029 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.172923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60298)
0929 06:33:36.172972 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.173002 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 06:33:36.173130 3580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37705: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
Entry for principal kudu/127.13.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab.
I20260929 06:33:36.185218 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37705
--webserver_interface=127.13.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:33:36.222447 3580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } attempt: 1
I20260929 06:33:36.222963 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.222814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60308)
0929 06:33:36.222884 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.222934 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 06:33:36.223282 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.223192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60310)
0929 06:33:36.223236 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.223259 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 06:33:36.223369 3580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37705: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
W20260929 06:33:36.263885 3587 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:36.264111 3587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:36.264139 3587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:36.264195 3587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:36.265326 3587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:36.265386 3587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:36.265412 3587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:36.265443 3587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:36.265472 3587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:36.266959 3587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.189.keytab
--principal=kudu/127.13.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.189:37705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:36.267213 3587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:36.267427 3587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:36.271946 3593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.272023 3595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.271962 3592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:36.272121 3587 server_base.cc:1034] running on GCE node
I20260929 06:33:36.272390 3587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:36.272675 3587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:36.273836 3587 hybrid_clock.cc:648] HybridClock initialized: now 1790663616273820 us; error 29 us; skew 500 ppm
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:36.276289 3587 init.cc:377] Logged in from keytab as kudu/127.13.15.189@KRBTEST.COM (short username kudu)
I20260929 06:33:36.276703 3587 webserver.cc:460] Webserver started at http://127.13.15.189:33393/ using document root <none> and password file <none>
I20260929 06:33:36.276881 3587 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:36.276937 3587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:36.277017 3587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:36.277905 3587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data/instance:
uuid: "19e51a353d3148898f0dbad8b92e99dd"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "4d7d0f7fb72866316780f0a40132b796"
server_key_iv: "02d29a686b61cc8252c9c9e286b8d1c3"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.278256 3587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal/instance:
uuid: "19e51a353d3148898f0dbad8b92e99dd"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "4d7d0f7fb72866316780f0a40132b796"
server_key_iv: "02d29a686b61cc8252c9c9e286b8d1c3"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.279453 3587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:36.280100 3602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.280241 3587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.280313 3587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal
uuid: "19e51a353d3148898f0dbad8b92e99dd"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "4d7d0f7fb72866316780f0a40132b796"
server_key_iv: "02d29a686b61cc8252c9c9e286b8d1c3"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.280360 3587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:36.289497 3587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:36.291136 3587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:36.291234 3587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:36.291513 3580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } attempt: 2
I20260929 06:33:36.291920 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.291766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60324)
0929 06:33:36.291851 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.291891 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 06:33:36.292325 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.292254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60332)
0929 06:33:36.292283 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.292308 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260929 06:33:36.292441 3580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.189:37705: Network error: Client connection negotiation failed: client connection to 127.13.15.189:37705: connect: Connection refused (error 111)
I20260929 06:33:36.295316 3587 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.189:37705
I20260929 06:33:36.295362 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.189:37705 every 8 connection(s)
I20260929 06:33:36.295692 3587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/data/info.pb
I20260929 06:33:36.296689 3655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:36.298220 3655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.299577 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3587
I20260929 06:33:36.299674 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-1/wal/instance
I20260929 06:33:36.299894 13374 external_mini_cluster.cc:1428] Setting key 675725559d024c1b4daada8e2b189dbc
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.15.188@KRBTEST.COM; defaulting to no policy
I20260929 06:33:36.307462 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.298852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.13.15.190:45203)
0929 06:33:36.298895 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.298897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.299126 (+ 229us) server_negotiation.cc:407] Connection header received
0929 06:33:36.299610 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.299611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.299621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.299724 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.300573 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.301047 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.301996 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.302279 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.302333 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.304613 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.304614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.304615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.304622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.305997 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.306491 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.306493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.306493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.306551 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.306796 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.306797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.306797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.306902 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.306966 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.307303 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.307386 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:36.307626 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.298846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45203 (local address 127.0.0.1:51810)
0929 06:33:36.299006 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.299037 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.299510 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.299838 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.299842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.300210 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.300478 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.300484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.301145 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.301146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.301919 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.301921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.302386 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.302387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.302552 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.302978 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.302988 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.304508 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.306165 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.306171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.306173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.306403 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.306617 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.306618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.306619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.306723 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.306971 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.306977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.307133 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.307280 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.307427 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260929 06:33:36.308092 3655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.15.188@KRBTEST.COM" created.
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:36.315567 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.308419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.13.15.189:37705)
0929 06:33:36.308556 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.308559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.308573 (+ 14us) server_negotiation.cc:407] Connection header received
0929 06:33:36.308587 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.308589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.308598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.308673 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.308916 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.309655 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.310617 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.310902 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.310977 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.312671 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.312677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.312677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.312682 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.314332 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.314709 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.314711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.314712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.314760 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.314980 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.314980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.314980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.315097 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.315155 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.315323 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.315448 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260929 06:33:36.315572 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.308377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60336)
0929 06:33:36.308416 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.308425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.308459 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.308674 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.308674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.308802 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.308840 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.308841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.309743 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.309744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.310537 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.310540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.310992 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.310993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.311073 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.311401 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.311407 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.312596 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.314406 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.314408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.314408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.314631 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.314843 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.314844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.314845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.314914 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.315165 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.315165 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.315211 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.315302 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.315402 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 06:33:36.316211 3655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.316488 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.316405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56004)
0929 06:33:36.316436 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.316474 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260929 06:33:36.316633 3604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
I20260929 06:33:36.316998 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.316938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56012)
0929 06:33:36.316962 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.316984 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260929 06:33:36.317112 3655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34579: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
Entry for principal kudu/127.13.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab.
I20260929 06:33:36.325006 13374 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJYAK93/build/release/bin/kudu
/tmp/dist-test-taskJYAK93/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34579
--webserver_interface=127.13.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.15.148:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 06:33:36.366329 3655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } attempt: 1
I20260929 06:33:36.366827 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.366663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56026)
0929 06:33:36.366761 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.366796 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 06:33:36.367187 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.367092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56042)
0929 06:33:36.367136 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.367160 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:33:36.367293 3655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34579: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
I20260929 06:33:36.383606 3580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } attempt: 3
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:36.390456 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.384043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.13.15.189:37705)
0929 06:33:36.384110 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.384112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.384123 (+ 11us) server_negotiation.cc:407] Connection header received
0929 06:33:36.384172 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.384173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.384186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.384258 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.384586 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.385097 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.385810 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.386037 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.386127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.387984 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.387985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.387986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.387993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.389338 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.389693 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.389694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.389694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.389744 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.389928 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.389929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.389930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.390033 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.390099 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.390291 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.390363 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 06:33:36.390465 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.383962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60344)
0929 06:33:36.384022 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.384034 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.384107 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.384264 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.384265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.384435 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.384499 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.384501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.385190 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.385192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.385725 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.385727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.386151 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.386153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.386232 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.386706 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.386711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.387877 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.389424 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.389425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.389426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.389617 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.389805 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.389806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.389806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.389861 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.390101 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.390102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.390183 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.390268 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.390364 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 06:33:36.390836 3580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.391144 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.391063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56048)
0929 06:33:36.391095 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.391121 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260929 06:33:36.391430 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.391340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56056)
0929 06:33:36.391380 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.391409 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 06:33:36.391522 3580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34579: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
W20260929 06:33:36.403343 3662 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 06:33:36.403589 3662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 06:33:36.403643 3662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 06:33:36.403685 3662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 06:33:36.404778 3662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 06:33:36.404840 3662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 06:33:36.404867 3662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 06:33:36.404893 3662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 06:33:36.404924 3662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 06:33:36.406404 3662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.15.148:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.15.190:45203,127.13.15.189:37705,127.13.15.188:34579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/krb5kdc/kudu_127.13.15.188.keytab
--principal=kudu/127.13.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.15.188:34579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 05:41:51 UTC on ccycloud.quasar-zzupah.root.comops.site
build id 4679345
I20260929 06:33:36.406669 3662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 06:33:36.406886 3662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 06:33:36.411607 3668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.411633 3667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 06:33:36.411633 3670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 06:33:36.411897 3662 server_base.cc:1034] running on GCE node
I20260929 06:33:36.412176 3662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 06:33:36.412459 3662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 06:33:36.413628 3662 hybrid_clock.cc:648] HybridClock initialized: now 1790663616413623 us; error 53 us; skew 500 ppm
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 06:33:36.416162 3662 init.cc:377] Logged in from keytab as kudu/127.13.15.188@KRBTEST.COM (short username kudu)
I20260929 06:33:36.416637 3662 webserver.cc:460] Webserver started at http://127.13.15.188:43963/ using document root <none> and password file <none>
I20260929 06:33:36.416857 3662 fs_manager.cc:359] Metadata directory not provided
I20260929 06:33:36.416923 3662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 06:33:36.417075 3662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 06:33:36.418076 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data/instance:
uuid: "77e7224488bd4bbe90b9350e5ef91d53"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "c56fd08d86cd2e9cd5c1a60bd35d89f7"
server_key_iv: "0827e648d90f483be4faf0893cf7d2b1"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.418478 3662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal/instance:
uuid: "77e7224488bd4bbe90b9350e5ef91d53"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "c56fd08d86cd2e9cd5c1a60bd35d89f7"
server_key_iv: "0827e648d90f483be4faf0893cf7d2b1"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.419695 3662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 06:33:36.420358 3677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.420523 3662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 06:33:36.420614 3662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data,/tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal
uuid: "77e7224488bd4bbe90b9350e5ef91d53"
format_stamp: "Formatted at 2026-09-29 06:33:36 on dist-test-slave-sm0p"
server_key: "c56fd08d86cd2e9cd5c1a60bd35d89f7"
server_key_iv: "0827e648d90f483be4faf0893cf7d2b1"
server_key_version: "encryptionkey@0"
I20260929 06:33:36.420696 3662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 06:33:36.422646 3580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } attempt: 1
I20260929 06:33:36.423000 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.422878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56068)
0929 06:33:36.422934 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.422966 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 06:33:36.423305 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.423206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56072)
0929 06:33:36.423250 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.423274 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 06:33:36.423391 3580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34579: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
I20260929 06:33:36.435433 3655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } attempt: 2
I20260929 06:33:36.435906 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.435753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56074)
0929 06:33:36.435828 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.435868 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 06:33:36.436305 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.436202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56082)
0929 06:33:36.436260 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.436285 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 06:33:36.436421 3655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.15.188:34579: Network error: Client connection negotiation failed: client connection to 127.13.15.188:34579: connect: Connection refused (error 111)
I20260929 06:33:36.437290 3662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 06:33:36.439594 3662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 06:33:36.439772 3662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 06:33:36.443970 3662 rpc_server.cc:307] RPC server started. Bound to: 127.13.15.188:34579
I20260929 06:33:36.444015 3729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.15.188:34579 every 8 connection(s)
I20260929 06:33:36.444397 3662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/data/info.pb
I20260929 06:33:36.445374 3730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 06:33:36.446911 3730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.449290 13374 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJYAK93/build/release/bin/kudu as pid 3662
I20260929 06:33:36.449398 13374 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJYAK93/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790663323716959-13374-0/minicluster-data/master-2/wal/instance
I20260929 06:33:36.449602 13374 external_mini_cluster.cc:1428] Setting key ef45faa7ace704b6ffeb8c21f977a3dd
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:33:36.456736 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.447581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.13.15.190:45203)
0929 06:33:36.447639 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.447641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.447856 (+ 215us) server_negotiation.cc:407] Connection header received
0929 06:33:36.448393 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.448394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.448403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.448457 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.449175 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.449784 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.451419 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.451619 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.451703 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.453905 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.453907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.453907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.453914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.455338 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.455761 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.455763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.455763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.455822 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.456079 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.456080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.456080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.456223 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.456306 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.456565 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.456646 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 06:33:36.456923 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.447535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.190:45203 (local address 127.0.0.1:51826)
0929 06:33:36.447709 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.447751 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.448307 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.448571 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.448575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.448822 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.449022 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.449027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.449883 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.449884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.451322 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.451325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.451734 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.451735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.451870 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.452386 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.452394 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.453808 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.455428 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.455435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.455437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.455681 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.455892 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.455892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.455893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.456002 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.456298 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.456304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.456438 (+ 134us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.456544 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.456692 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260929 06:33:36.457491 3730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.458693 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.449967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.13.15.188:34579)
0929 06:33:36.450226 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.450230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.450330 (+ 100us) server_negotiation.cc:407] Connection header received
0929 06:33:36.450369 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.450370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.450386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.450448 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.451280 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.451993 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.452874 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.453175 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.453267 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.455515 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.455522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.455523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.455529 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.457389 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.457778 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.457781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.457781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.457842 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.458071 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.458072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.458073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.458267 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.458423 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.458489 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.458572 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.190@KRBTEST.COM
I20260929 06:33:36.464389 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.457743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.189:37705 (local address 127.0.0.1:60360)
0929 06:33:36.457808 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.457818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.457854 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.457987 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.457988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.458115 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.458183 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.458184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.459010 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.459010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.459677 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.459679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.460042 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.460042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.460098 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.460471 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.460475 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.461667 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.463442 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.463442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.463443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.463615 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.463821 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.463822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.463822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.463882 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.464092 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.464093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.464136 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.464212 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.464299 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 06:33:36.464540 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.457805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.13.15.189:37705)
0929 06:33:36.457850 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.457851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.457859 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:36.457917 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.457918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.457928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.457983 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.458247 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.458707 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.459765 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.459970 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.460045 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.461749 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.461750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.461750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.461757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.463343 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.463706 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.463707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.463707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.463756 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.463945 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.463945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.463945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.464032 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.464077 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.464232 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.464421 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:33:36.464838 3730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } has no permanent_uuid. Determining permanent_uuid...
I20260929 06:33:36.467381 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.459470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.13.15.190:45203)
0929 06:33:36.459505 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.459507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.459515 (+ 8us) server_negotiation.cc:407] Connection header received
0929 06:33:36.459557 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.459558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.459567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.459620 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.460064 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.460670 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.461631 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.461922 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.462002 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.464760 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.464761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.464762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.464769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.466347 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.466642 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.466643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.466644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.466699 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.466868 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.466869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.466869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.466991 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.467124 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.467272 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.467327 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.188@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.15.189@KRBTEST.COM
I20260929 06:33:36.472365 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.465077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56088)
0929 06:33:36.465153 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.465162 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.465197 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.465350 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.465351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.465493 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.465530 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.465531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.466226 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.466227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.466963 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.466965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.467533 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.467534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.467580 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.468043 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.468048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.469465 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.471373 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.471374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.471375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.471539 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.471760 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.471760 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.471760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.471818 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.472030 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.472031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.472075 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.472154 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.472239 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 06:33:36.472397 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.465142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.13.15.188:34579)
0929 06:33:36.465176 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.465178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.465187 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:36.465278 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.465279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.465287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.465335 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.465602 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.466113 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.467090 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.467406 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.467480 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.469554 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.469555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.469555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.469560 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.471271 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.471619 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.471620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.471621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.471667 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.471874 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.471875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.471875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.471965 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.472015 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.472170 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.472277 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":18}
I20260929 06:33:36.473855 3730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: Bootstrap starting.
I20260929 06:33:36.474690 3730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:36.475040 3730 log.cc:824] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:36.475668 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.468097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.13.15.189:37705)
0929 06:33:36.468134 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.468136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.468145 (+ 9us) server_negotiation.cc:407] Connection header received
0929 06:33:36.468201 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.468202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.468210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.468255 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.468749 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.469464 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.470392 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.470615 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.470696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.472948 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.472949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.472949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.472955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.474362 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.474697 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.474698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.474698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.474748 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.474990 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.474991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.474992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.475131 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.475297 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.475493 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.475566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 06:33:36.475885 3730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: No bootstrap required, opened a new log
I20260929 06:33:36.477056 13374 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 06:33:36.477825 3730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.477947 3730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e7224488bd4bbe90b9350e5ef91d53, State: Initialized, Role: FOLLOWER
I20260929 06:33:36.478030 3730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.478363 3730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:36.478327 3736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.478472 3736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.478261 3554 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fface9f866124414911200e23da68b6a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51830
W20260929 06:33:36.480057 3747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:33:36.480119 3747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:36.480527 3662 master_runner.cc:186] Error getting master registration for 127.13.15.190:45203: OK, instance_id { permanent_uuid: "fface9f866124414911200e23da68b6a" instance_seqno: 1790663616156118 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:33:36.483384 3662 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:33:36.498529 3580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } attempt: 2
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.190@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:33:36.506901 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.498920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56102)
0929 06:33:36.498982 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.498996 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.499059 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.499256 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.499257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.499478 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.499534 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.499536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.500459 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.500461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.501284 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.501285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.501766 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.501767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.501826 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.502385 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.502390 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.503939 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.505741 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.505742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.505743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.505968 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.506193 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.506193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.506194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.506263 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.506595 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.506596 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.506641 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.506713 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.506804 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 06:33:36.506914 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.499003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.13.15.188:34579)
0929 06:33:36.499078 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.499081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.499108 (+ 27us) server_negotiation.cc:407] Connection header received
0929 06:33:36.499138 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.499140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.499150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.499221 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.499619 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.500378 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.501375 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.501683 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.501758 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.504039 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.504040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.504040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.504047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.505652 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.506046 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.506047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.506047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.506101 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.506328 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.506328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.506329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.506526 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.506589 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.506750 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.506823 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 06:33:36.508514 3580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: Bootstrap starting.
I20260929 06:33:36.509277 3580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:36.509608 3580 log.cc:824] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:36.510442 3580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: No bootstrap required, opened a new log
I20260929 06:33:36.511734 3580 raft_consensus.cc:348] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.511873 3580 raft_consensus.cc:729] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fface9f866124414911200e23da68b6a, State: Initialized, Role: FOLLOWER
I20260929 06:33:36.511941 3580 consensus_queue.cc:260] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.512261 3580 sys_catalog.cc:564] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:36.512313 3554 raft_consensus.cc:482] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 06:33:36.512355 3554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:36.513176 3554 raft_consensus.cc:504] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.513439 3554 leader_election.cc:288] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [CANDIDATE]: Term 1 election: Requested vote from peers 19e51a353d3148898f0dbad8b92e99dd (127.13.15.189:37705), 77e7224488bd4bbe90b9350e5ef91d53 (127.13.15.188:34579)
I20260929 06:33:36.512204 3750 sys_catalog.cc:455] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.513814 3750 sys_catalog.cc:458] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.514842 3704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fface9f866124414911200e23da68b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "77e7224488bd4bbe90b9350e5ef91d53"
I20260929 06:33:36.514959 3704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [term 0 FOLLOWER]: Advancing to term 1
W20260929 06:33:36.515271 3765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 06:33:36.515314 3765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:36.515828 3704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fface9f866124414911200e23da68b6a in term 1.
I20260929 06:33:36.516074 3529 leader_election.cc:302] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e7224488bd4bbe90b9350e5ef91d53, fface9f866124414911200e23da68b6a; no voters:
I20260929 06:33:36.516189 3750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 06:33:36.516325 3750 raft_consensus.cc:686] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [term 1 LEADER]: Becoming Leader. State: Replica: fface9f866124414911200e23da68b6a, State: Running, Role: LEADER
I20260929 06:33:36.516407 3750 consensus_queue.cc:237] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.516796 3750 sys_catalog.cc:455] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fface9f866124414911200e23da68b6a. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.516899 3750 sys_catalog.cc:458] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:36.517100 3770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 06:33:36.517287 3770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 06:33:36.515740 3512 master_runner.cc:186] Error getting master registration for 127.13.15.189:37705: OK, instance_id { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" instance_seqno: 1790663616294210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 06:33:36.517997 3512 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 06:33:36.527541 3655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } attempt: 3
I20260929 06:33:36.528983 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.517253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.13.15.189:37705)
0929 06:33:36.517295 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.517297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.518125 (+ 828us) server_negotiation.cc:407] Connection header received
0929 06:33:36.518293 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.518294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.518305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.518368 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.519512 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.520288 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.521731 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.522558 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.522602 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.525026 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.525040 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.525041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.525049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.526697 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.527270 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.527272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.527272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.527340 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.528228 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.528229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.528230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.528372 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.528442 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.528798 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.528876 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 06:33:36.530324 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.516561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.13.15.190:45203)
0929 06:33:36.516601 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.516603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.518000 (+ 1397us) server_negotiation.cc:407] Connection header received
0929 06:33:36.518565 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.518566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.518577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.518607 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.519807 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.520473 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.523231 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.523553 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.523610 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.526448 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.526449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.526449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.526456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.528301 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.529361 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.529362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.529363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.529428 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.529760 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.529761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.529762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.529876 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.529962 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.530157 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.530224 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:36.531095 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.517694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56110 (local address 127.13.15.188:34579)
0929 06:33:36.517729 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.517731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.518396 (+ 665us) server_negotiation.cc:407] Connection header received
0929 06:33:36.518494 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.518495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.518504 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.518533 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.521475 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.522197 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.523948 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.524286 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.524331 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.526744 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.526745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.526746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.526752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.528990 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.529689 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.529690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.529690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.529763 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.530453 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.530454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.530455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.530593 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.530677 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.530914 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.530984 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 06:33:36.532357 3704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [term 1 FOLLOWER]: Refusing update from remote peer fface9f866124414911200e23da68b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:36.532657 3767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:36.534746 3736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fface9f866124414911200e23da68b6a. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.534833 3736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: This master's current role is: FOLLOWER
Sep 29 06:33:36 dist-test-slave-sm0p krb5kdc[3499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790663616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.15.189@KRBTEST.COM for kudu/127.13.15.188@KRBTEST.COM
I20260929 06:33:36.538434 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.527780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.15.188:34579 (local address 127.0.0.1:56116)
0929 06:33:36.527856 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 06:33:36.527866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 06:33:36.527902 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 06:33:36.531293 (+ 3391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 06:33:36.531295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 06:33:36.531452 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 06:33:36.531494 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.531496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.532484 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.532485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.533216 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 06:33:36.533218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.533659 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.533660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 06:33:36.533760 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.534196 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 06:33:36.534201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 06:33:36.535670 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 06:33:36.537391 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.537391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.537392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.537592 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.537799 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 06:33:36.537800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 06:33:36.537800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 06:33:36.537851 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 06:33:36.538117 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 06:33:36.538117 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 06:33:36.538189 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 06:33:36.538251 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 06:33:36.538336 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 06:33:36.538659 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:36.527977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.13.15.188:34579)
0929 06:33:36.531165 (+ 3188us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.531167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.531174 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:33:36.531192 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.531193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.531202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.531251 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.531719 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.532378 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.533301 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:36.533546 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:36.533599 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:36.535772 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 06:33:36.535773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 06:33:36.535773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 06:33:36.535779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 06:33:36.537315 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.537678 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.537679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.537679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.537741 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 06:33:36.537936 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 06:33:36.537937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 06:33:36.537937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 06:33:36.538045 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 06:33:36.538095 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:36.538301 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 06:33:36.538430 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3173,"thread_start_us":76,"threads_started":1}
I20260929 06:33:36.539994 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: Bootstrap starting.
I20260929 06:33:36.540577 3655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: Neither blocks nor log segments found. Creating new log.
I20260929 06:33:36.540855 3655 log.cc:824] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: Log is configured to *not* fsync() on all Append() calls
I20260929 06:33:36.541608 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: No bootstrap required, opened a new log
I20260929 06:33:36.543766 3750 sys_catalog.cc:455] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.543953 3750 sys_catalog.cc:458] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:36.543936 3655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.544137 3655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e51a353d3148898f0dbad8b92e99dd, State: Initialized, Role: FOLLOWER
I20260929 06:33:36.544276 3655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } }
I20260929 06:33:36.544799 3627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [term 0 FOLLOWER]: Advancing to term 1
I20260929 06:33:36.544966 3736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.545068 3736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.545434 3629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fface9f866124414911200e23da68b6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "19e51a353d3148898f0dbad8b92e99dd"
I20260929 06:33:36.545703 3627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [term 1 FOLLOWER]: Refusing update from remote peer fface9f866124414911200e23da68b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 06:33:36.545855 3773 mvcc.cc:205] Tried to move back new op lower bound from 7334558173312147456 to 7334558173251846144. Current Snapshot: MvccSnapshot[applied={T|T < 7334558173312147456}]
I20260929 06:33:36.546452 3767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 06:33:36.547434 3629 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fface9f866124414911200e23da68b6a in term 1.
I20260929 06:33:36.548902 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.549007 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.549065 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fface9f866124414911200e23da68b6a. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.549114 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.549149 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.549211 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 06:33:36.544725 3655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 06:33:36.554852 3770 catalog_manager.cc:1347] Generated new cluster ID: 10b609d938964e16a2d5a11669432243
I20260929 06:33:36.554909 3770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 06:33:36.551268 3797 catalog_manager.cc:1259] Loaded cluster ID: 10b609d938964e16a2d5a11669432243
I20260929 06:33:36.558182 3797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: loading cluster ID for follower catalog manager: success
W20260929 06:33:36.558506 3797 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260929 06:33:36.558579 3797 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260929 06:33:36.570658 3767 sys_catalog.cc:455] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fface9f866124414911200e23da68b6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fface9f866124414911200e23da68b6a" member_type: VOTER last_known_addr { host: "127.13.15.190" port: 45203 } } peers { permanent_uuid: "19e51a353d3148898f0dbad8b92e99dd" member_type: VOTER last_known_addr { host: "127.13.15.189" port: 37705 } } peers { permanent_uuid: "77e7224488bd4bbe90b9350e5ef91d53" member_type: VOTER last_known_addr { host: "127.13.15.188" port: 34579 } } }
I20260929 06:33:36.570781 3767 sys_catalog.cc:458] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a [sys.catalog]: This master's current role is: LEADER
I20260929 06:33:36.573114 3770 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 06:33:36.574194 3770 catalog_manager.cc:1504] Loading token signing keys...
I20260929 06:33:36.581025 3770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fface9f866124414911200e23da68b6a: Generated new TSK 0
I20260929 06:33:36.581214 3770 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 06:33:36.601392 3583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.599871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.13.15.190:45203)
0929 06:33:36.599913 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.599915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.600805 (+ 890us) server_negotiation.cc:407] Connection header received
0929 06:33:36.600899 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.600900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.600910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.600937 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.601332 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51852: BlockingRecv error: recv got EOF from 127.0.0.1:51852 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 06:33:36.602110 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.600245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.13.15.189:37705)
0929 06:33:36.600282 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.600283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.601627 (+ 1344us) server_negotiation.cc:407] Connection header received
0929 06:33:36.601735 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.601735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.601745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.601773 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.602052 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60394: BlockingRecv error: recv got EOF from 127.0.0.1:60394 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 06:33:36.602794 3734 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.600535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.13.15.188:34579)
0929 06:33:36.600571 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.600573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.602331 (+ 1758us) server_negotiation.cc:407] Connection header received
0929 06:33:36.602424 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.602425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.602434 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.602462 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.602737 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56126: BlockingRecv error: recv got EOF from 127.0.0.1:56126 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 06:33:36.604970 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.604258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.13.15.189:37705)
0929 06:33:36.604290 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.604291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.604481 (+ 190us) server_negotiation.cc:407] Connection header received
0929 06:33:36.604580 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.604581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.604597 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.604625 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.604930 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60410: BlockingRecv error: recv got EOF from 127.0.0.1:60410 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 06:33:36.607781 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.607108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.13.15.189:37705)
0929 06:33:36.607137 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.607138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.607347 (+ 209us) server_negotiation.cc:407] Connection header received
0929 06:33:36.607428 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.607429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.607437 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.607461 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.607744 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60422: BlockingRecv error: recv got EOF from 127.0.0.1:60422 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 06:33:36.611527 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.610888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.13.15.189:37705)
0929 06:33:36.610917 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.610919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.611103 (+ 184us) server_negotiation.cc:407] Connection header received
0929 06:33:36.611186 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.611186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.611195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.611221 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.611487 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60424: BlockingRecv error: recv got EOF from 127.0.0.1:60424 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 06:33:36.617375 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.616601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.13.15.189:37705)
0929 06:33:36.616633 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.616634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.616898 (+ 264us) server_negotiation.cc:407] Connection header received
0929 06:33:36.616992 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.616992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.617002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.617029 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.617334 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60436: BlockingRecv error: recv got EOF from 127.0.0.1:60436 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 06:33:36.627240 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.626455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.13.15.189:37705)
0929 06:33:36.626486 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.626488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.626747 (+ 259us) server_negotiation.cc:407] Connection header received
0929 06:33:36.626843 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.626844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.626854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.626882 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.627196 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60438: BlockingRecv error: recv got EOF from 127.0.0.1:60438 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 06:33:36.645365 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.644422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.13.15.189:37705)
0929 06:33:36.644462 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.644464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.644785 (+ 321us) server_negotiation.cc:407] Connection header received
0929 06:33:36.644911 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.644912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.644924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.644954 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.645314 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60450: BlockingRecv error: recv got EOF from 127.0.0.1:60450 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 06:33:36.679512 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.678711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.13.15.189:37705)
0929 06:33:36.678751 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.678753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.678964 (+ 211us) server_negotiation.cc:407] Connection header received
0929 06:33:36.679006 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.679007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.679020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.679086 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.679459 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60456: BlockingRecv error: recv got EOF from 127.0.0.1:60456 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 06:33:36.745188 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.744583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.13.15.189:37705)
0929 06:33:36.744647 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.744650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.744682 (+ 32us) server_negotiation.cc:407] Connection header received
0929 06:33:36.744759 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.744760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.744771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.744822 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.745142 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60460: BlockingRecv error: recv got EOF from 127.0.0.1:60460 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 06:33:36.875129 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:36.874471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.13.15.189:37705)
0929 06:33:36.874640 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:36.874642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:36.874678 (+ 36us) server_negotiation.cc:407] Connection header received
0929 06:33:36.874702 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:36.874703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:36.874713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:36.874760 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:36.875062 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60466: BlockingRecv error: recv got EOF from 127.0.0.1:60466 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260929 06:33:37.132972 3659 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 06:33:37.132360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.13.15.189:37705)
0929 06:33:37.132433 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.132435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.132476 (+ 41us) server_negotiation.cc:407] Connection header received
0929 06:33:37.132502 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.132503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.132512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.132555 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 06:33:37.132928 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60480: BlockingRecv error: recv got EOF from 127.0.0.1:60480 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 06:33:37.480526 3747 catalog_manager.cc:1259] Loaded cluster ID: 10b609d938964e16a2d5a11669432243
I20260929 06:33:37.480592 3747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: loading cluster ID for follower catalog manager: success
I20260929 06:33:37.481376 3747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: acquiring CA information for follower catalog manager: success
I20260929 06:33:37.481739 3747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77e7224488bd4bbe90b9350e5ef91d53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:33:37.560142 3797 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: acquiring CA information for follower catalog manager: success
I20260929 06:33:37.560616 3797 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19e51a353d3148898f0dbad8b92e99dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 06:33:37.648792 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:37.646325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.13.15.189:37705)
0929 06:33:37.646448 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.646450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.646457 (+ 7us) server_negotiation.cc:407] Connection header received
0929 06:33:37.646481 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.646482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.646530 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.646609 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:37.646772 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.647255 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.647982 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.648172 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.648260 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:37.648424 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:37.648557 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:37.648630 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:37.648675 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 06:33:37.648696 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":51,"threads_started":1}
I20260929 06:33:37.652807 3870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:37.650606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.13.15.188:34579)
0929 06:33:37.650744 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.650746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.650846 (+ 100us) server_negotiation.cc:407] Connection header received
0929 06:33:37.650904 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.650905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.650938 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.650999 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:37.651081 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.651505 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.652174 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.652350 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.652396 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:37.652497 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:37.652592 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:37.652643 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:37.652677 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 06:33:37.652691 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":53,"threads_started":1}
I20260929 06:33:37.656932 3870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:37.654706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56152 (local address 127.13.15.188:34579)
0929 06:33:37.654751 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.654753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.654793 (+ 40us) server_negotiation.cc:407] Connection header received
0929 06:33:37.654853 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.654854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.654863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.654910 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:37.655049 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.655539 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.656281 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.656474 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.656543 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:37.656702 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:37.656731 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:37.656782 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:37.656839 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 06:33:37.656857 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 06:33:37.657603 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:37.654409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.13.15.189:37705)
0929 06:33:37.654459 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.654461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.654780 (+ 319us) server_negotiation.cc:407] Connection header received
0929 06:33:37.654847 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.654849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.654860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.654918 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:37.655101 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.655563 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.656645 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.656939 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.656997 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:37.657329 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:37.657365 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:37.657436 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:37.657495 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 06:33:37.657515 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 06:33:37.658514 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 06:33:37.654412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.13.15.190:45203)
0929 06:33:37.654565 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 06:33:37.654567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 06:33:37.654660 (+ 93us) server_negotiation.cc:407] Connection header received
0929 06:33:37.654921 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 06:33:37.654922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 06:33:37.654972 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 06:33:37.655049 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 06:33:37.655772 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.656519 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.657503 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 06:33:37.657783 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 06:33:37.657851 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 06:33:37.658063 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 06:33:37.658201 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 06:33:37.658263 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 06:33:37.658374 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 06:33:37.658393 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260929 06:33:37.667044 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3512
I20260929 06:33:37.672919 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3587
I20260929 06:33:37.677721 13374 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJYAK93/build/release/bin/kudu with pid 3662
2026-09-29T06:33:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2757 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5580 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293968 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS